BINTRACKER
February 03, 2023
Winter time is Bintracker time! After a busy summer and fall here on our homestead, I finally found some time to work on the project again. And this time around, I’ve got some pretty exiting news to share.
It’s been an extraordinarily challenging and exhausting process (more on that below), but it’s done: We have a working Windows build. Head over to https://github.com/bintracker/bintracker/releases to check it out. A little refresher on what’s currently possible in Bintracker:
Note that the Windows build does not contain the ROM files required by MAME - read this guide for details on how to add those.
Bintracker is definitely still rough around the edges, so be prepared for frequent crashes and other nasty surprises. Save often, and most importantly, please report any bugs you come across on the issue tracker or by sending a mail to the address you’re receiving this newsletter from.
Also, big thanks to AtariTufty, who was brave enough to test the pre-release.
Aside from getting Windows builds off the ground, I’ve mostly done bug fixes. Also, the latest mkdocs and mkdocs-material 9 broke the documentation build, so I spent some time on getting all of that up to date and running again. Still not 100% there yet - if anybody wants to look into customizing the color theme that does not involve hacking the main css, that would be great. The officially prescribed method of putting an extra.css in docs/stylesheets does zilch. In hindsight, I’m not sure if choosing mkdocs was the right decision, as this isn’t the first time an update broke the build. I hope to run everything off scm2wiki one day, but that day is still far off, unfortunately.
I also temporarily removed the PhaserX-MZ700 engine, as it has some stability issues that I haven’t been able to fix yet. Instead, I’ve added a new engine for the Fairchild Channel F. Sleizsa Quartet is the successor to the trusty Sleizsa Trio engine, featuring 4 channels, variable duty cycles, PWM sweep, and PWM sampled drums. I’ve had this laying around since 2019 - happy to be finally able to use it!
This is the part where I rant about the horrors of the whole Windows build experience, so feel free to skip ahead if you aren’t interested in the nitty gritty details.
My original plan was to use MinGW to cross-compile from Linux. After a lot of hunting down obscure dependencies, kicking and massaging the makefile, and switching to another PNG library because I couldn’t get MinGW to build imlib2, I actually managed to get the project to build. There was only one small problem - on Wine, the program would crash after startup, and on real Windows, it wouldn’t start at all. Some Chicken Scheme experts suggested to increase the executable heap size, which up to certain point made the program crash… slower. Beyond that point, Bintracker would run on Wine - but crash with a STATUS_BAD_STACK error on actual Windows. Who’d have thought that nowadays Wine is actually better at running dodgy Windows software that Windows itself?
Anyway, I abandoned cross-compilation and set up a Win10 virtual machine with VirtualBox instead. Now, using MinGW on Linux is not exactly pleasant, but compared to MinGW on Windows, it’s heaven. On Windows, nothing works out of the box. Installation doesn’t set up $PATH properly, and it’s not like simply adding MinGW’s bin directory will do. Also, why does MinGW on Windows have like 10 different shells? How am I supposed to know which one I need? Well, let’s have a look at MinGW’s documentation. Oh. Ok. I mean, I know Bintracker’s documentation isn’t exactly stellar at this point, but this… well, never mind. Long story short, eventually I managed to build Chicken Scheme. And sure enough - the interpreter works, but the compiler doesn’t, because something something not found error lol.
Tired of trying to coerce plain MinGW into producing a workable Chicken compiler, my next step was to try and build Chicken on MSYS (using MinGW under the hood, of course). My memory of this is hazy - in any case, it didn’t work. Then, while searching for solutions on the web, I stumbled across Chicken Scheme the Chocolatey package. I had nothing to lose at this point, so I gave it a try. And it actually worked. Sure, it’s an old Chicken 5.2, but hey, I’ll take it. Massive props to package maintainer zilti. From here on, it was almost smooth sailing for a while. SQLite installed like a charm in MSYS. Chicken’s SRFI-18 multi-threading library wouldn’t budge at first, but I managed to get it to work by tweaking its build script at bit. And at last, I was finally able to build Bintracker again. Even better, that build would actually start up without complaints.
Of course, I wasn’t done yet - pop-up dialogues weren’t working, and once I fixed that, it turned out that the way Bintracker was communicating with MAME under Linux wouldn’t be possible under Windows. Around that time, the VM started to act up as well, complaining about hard drive failures (yeah right) and eventually refusing to boot. Farming season was starting up by then as well, so I put the project on hold for the time being.
When I returned to it in November, I could no longer get a working Windows VM in VirtualBox. Tried VMware, alas without much success, as I’m on a non-systemd Linux distro, and VMware is pretty much married to systemd nowadays. I looked into setting up a VM with KVM/QEMU, but MS doesn’t provide any evaluation images for this setup, and at my income levels I’m not inclined to shell out money for a full Windows license. So I was about to give up when I stumbled across Vagrant, which can run on top of KVM/QEMU and has a Win10 VM image in its catalog. And holy cow, does it work smoothly. Traumatized by VirtualBox, I didn’t dare to imagine that virtualization could ever be pleasant. No random crashes, no wonky proprietary “Guest Additions” (not that it matters much on a proprietary operating system), just rdesktop and rsync and off you go. Brilliant.
So, off I go installing the toolchain via Chocolatey again, installing the libraries, when suddenly -whoops- the check-errors library won’t build and actually completely borks the chicken-install utility in the process. How come that didn’t happen earlier? Turns out check-errors got an update in the meantime, which for some reason doesn’t work under Windows. It’s a low-level dependency that I definitely can’t do without in Bintracker, so I needed to fix this somehow. Took me several days to find the problem - the library actually builds and installs fine, but fails the final step of creating an .egg-info file, which consequently sends chicken-install to chicken-hell. After manually creating that file in the appropriate location, everything works as intended again.
Surely, our odyssey must be over soon. Just need to figure out a way to talk to MAME on Windows… So how about using TCP? I’ve been meaning to add TCP support anyway, as this would open up the path to supporting some emulators other than MAME. Chicken Scheme comes with a TCP module, so implementing TCP on Bintracker’s side was easy enough. Unfortunately, doing it on MAME’s side proved to a little more… challenging (for the inept developer that I am, in any case). MAME scripting happens in Lua, and in order to make Lua speak TCP, you need the luasocket library. And in order to build that, you first need Lua on your system. Hoping I could spare myself the trouble, I hunted around for binary builds of the library, which are surprisingly hard to come by. Eventually I found some, but they were built with Lua 5.1, whereas MAME runs Lua 5.3. So, I took a deep breath, set up a Lua toolchain, and, after a bit of makefile massaging, was able to build the lib. Just a slight problem - when importing the library into MAME’s Lua interpreter, it goes looking for either a core.dll or a socket.dll. Great, except that’s not what the luasocket build outputs. Can’t remember everything that transpired afterwards, but rest assured I tried a great many things, all of which ended in some variation of the by now all-too-familiar “module socket.core not found” error. Eventually I went hunting for binaries again and this time managed to find an actual Lua 5.3 build. Did it work? Well, take a guess. Turns out, you don’t just need to build against the correct Lua main version, you need to build against the exact point release. Well, maybe, that is. I never managed to get luasocket working in MAME, but from reading various forum posts it appears that some people have (and a great many more failed). Either way, one day I happened to browse around in the source code of MAME’s Lua interpreter, when I stumbled across this gem:
* file:open(name) - open first file matching name in searchpath. supports read
* and write sockets as "socket.127.0.0.1:1234"
You’ve got to be kidding me! Indeed, as it turns out we don’t need luasocket to make MAME speak TCP after all. Now the ordeal is almost over. I made one more mistake that cost me another couple of days, likely due to my inexperience with networking shenanigans. My plan was to implement things so that MAME would be the client, and Bintracker would be the command&control server. My reasoning was that Bintracker has no means of knowing when MAME is done booting, so MAME should contact Bintracker once ready, and request further instructions. Turns out that this probably doesn’t work with MAME’s emu_file implementation - in any case, I couldn’t get it to talk to the listener on Bintracker’s side, even though I could connect to it with another client just fine. So instead I made Bintracker the client, and just let it mindlessly pound the listener on MAME’s side until that one finally answers. And with that, I finally had a working Windows build. Yay, that was fun. NOT. I guess an experienced developer could have figured this out in a day or two. Too bad I’m not an experienced developer, but just some guy trying to make music on some obscure 40 year old junk.
At this point Bintracker is still firmly in alpha stage. I would like to get the project to beta stage as soon as possible. Therefore, the plan for now is to hold off on implementing any major new features, and instead focus on improving the overall stability of Bintracker (and of course managing the onslaught of incoming bug reports from you all). That said, there are a number of things I would like to tackle eventually. In no particular order:
I also still need to set up continuous integration. Any help with that would be greatly appreciated. Not sure yet which route to take - I very much don’t want to rely on Github for this, but on the other hand Github Actions seems to be the easiest way nowadays. Anyway, we’ll see how far I get before the farming season starts up again. I will definitely spend more time on Bintracker in the coming weeks, but I also have a few other projects that have been on hold for much too long.
April 01, 2022
Again, a long time has passed since the last update. My apologies for keeping you waiting so long. Things got way to busy over here at some point while I was processing last year’s harvest, finishing my language classes, making a new album with my band Doppelplusungut, and working on various other side projects. To be honest I also got a bit stuck on that rewrite of the MDAL compiler I mentioned last time.
That said, here are the good news - for the past two months, I’ve been making steady progress again. I’ve had plenty of time to think about how to implement the new compiler, and in the end I’ve come up with a design that I believe to be sound and solid. And what’s even better - I’ve actually finished implementing it!
Before I delve a bit more into the details of the new compiler though, I’d like to take the opportunity to thank all the kind folks who’ve donated in the past months. Thank you all for continuing to believe in the project despite the lack of (public) activity.
Now, what’s the deal with this new compiler? Was it really worth sinking two months of coding work into this when we already had a working compiler before? The truth is that the old compiler had a major design flaw that would have made it impossible to achieve Bintracker’s goal of supporting any arbitrary sound engine, while also making it difficult to implement an efficient optimizer later on. Aside from that, as I mentioned last time, the old design had become quite a mess as I was essentially making things up on the fly.
The aforementioned design flaw has to do with the way references are resolved. The old compiler was tracking the output origin (ie. the address in memory where things get compiled to) during the whole compilation process, resolving references as early as possible. While this approach allows for a fast compilation, it makes it impossible to modify the generated data later on. Say we have an engine that uses samples, and the pattern data references those samples by their address. Now we want to eliminate unused samples from the output. In order to determine which samples are unused, we need to first evaluate the pattern data. Once the pattern data is evaluated, the sample references are now resolved to addresses. However, eliminating samples will change the addresses of the samples that come after the ones that have been eliminated. So now the references in the pattern data must be changed - but this is impossible, since they are already resolved, so they are now just numbers, indistinguishable from the rest of the data.
I guess this would have been obvious to any software developer who’s ever studied compiler design. Unfortunately, not being a “real” software developer myself, it wasn’t obvious to me. So, that was a rather costly mistake that was nevertheless bound to happen, as I was predominantly concerned with making the compiler as fast as possible.
The new MDAL compiler does not track the output origin at all, and does not resolve any references. Instead, the compiler generator first builds an assembler-level abstract syntax tree when reading a given MDAL engine definition. This AST is passed to the assembler, which will already translate as much as at it can to machine code. The result of this operation is cached. The compiler generator then eliminates all output nodes that require no further processing at the MDAL level, and reorders the remaining ones to minimize the number of compilation passes required. On compiling the actual module, the compiler stores the result of evaluating each node in a global symbol table. Bindings in this table can be updated throughout the compilation process, so in the process of evaluating one node, the result of another node can be modified. The final step of the compilation process, which includes reference resolution, is then offloaded to the one component of Bintracker that is good at resolving references - the assembler.
Regarding speed, the new compiler looks to be generally more efficient than the old one - unless there are a lot of references to be resolved. Currently, wiz4v is the one engine that suffers from this, now having a noticible lag when entering notes. I believe there is plenty of room for optimizing reference resolution though.
Overall, I’m very happy with the new design, and also much relieved that I finally managed to eliminate this massive roadblock. Not only did the rewrite solve some major problems, but the code has also become more readable in the process (though it still needs work in that respect). In total, the rewrite added just about 200 lines of code, while packing significantly more functionality than before.
While there are still a number of glaring bugs and missing features (still no Label commands, for instance), I think it’s time to tackle Windows builds next. There’s actually been some progress in this department - at least it’s possible to build Bintracker with MinGW now. Unfortunately that doesn’t mean that the builds are actually working, yet. On Wine, Bintracker starts up, but then hangs on opening a module. On actual Windows, it either crashes on startup or fails to start at all so far. Not sure how long this is going to take, so I’m not going to make any promises in terms of a release date just yet, especially considering this project’s history of missed deadlines.
As I’ve mentioned before, I don’t have much experience building stuff for Windows. Someone more knowledgeable could probably figure out things much quicker than me. In short, I could really use some help with this. If you’re up for some good old head-against-wall-banging and pulling-hair-in-frustration then please get in touch via the contact form.
June 06, 2021
Wow, it’s actually been over half a year since the last update. My sincere apologies for the long silence. Not only did the break I announced last time take longer than planned, thanks to a few unexpected side projects. It also turns out that getting our new homestead up and running takes up even more time than expected. Nevertheless, Bintracker is alive and kicking, and I’ve been making good progress in what little time I can spare.
Before we go any further, it’s time for a big shout-out to radian, who passed a sizeable donation my way a while back. Thanks a lot, mate!
A lot of the recent work has been focussed on improving stability and usability. A great many bugs were squashed, including a particularly nasty one that prevented spinboxes (those little selector widgets used to set the edit step, base octave etc.) from working properly. I’ve struggled with that one for almost a year. I’ve also made some minor tweaks to the GUI, and made the default key bindings a bit more intuitive.
Furthermore, MAME version 0.227 introduced breaking changes to the LUA engine. This affected emulation in Bintracker, since it relies on the LUA engine to control MAME (thanks Jeremy Penner for reporting this). While implementing a work-around that ensures compatibility with both old and new versions of MAME, I also improved the rest of the MAME bridge at bit. There’s still more work to be done, but at least I got rid of most of the audio glitches that where quite common until now.
Now for some big news: Support for unordered Groups is done. That means Bintracker can now handle things like effect tables, ornaments, samples, and wavetables. This was one of the biggest missing building blocks needed to complete the Bintracker core application. It was quite a tough nut to crack, but in the end I found a rather elegant solution that required just a few hundred lines of code. Long-winded explanation follows, feel free to skip ahead to the end of this section if you aren’t interested in the gory details.
Bintracker’s MDAL backend treats any kind of repeating structure (eg. patterns, tables, samples) as a Group node with one or more Block members. A Group may be ordered or unordered, in other words, a Group’s Block members may or may not be sequenced. Ordered Groups use an additional block member to control the order of the remaining block instances. A typical MDAL engine definition will have an ordered Group for (note) patterns, where each block member represents a channel/track, and the Order block represents the sequence/order list. But what does it even mean for a Group to be unordered?
One possible use case would be samples, which can be realized as an unordered Group with a single Block member holding the sample data. In theory, this Group does not need an order list, because we just want each sample added once to the output. However, Block instances in MDAL are either endless, or have a fixed length. So in practice our Group does need an Order block, to specify the length of the sample block instances. It is even conceivable (if slightly uncommon) that users would use the same sample twice, with different lengths.
This led me to the realization that the easiest way of implementing unordered Groups is to treat them exactly like ordered Groups, as far as input is concerned. As the old saying goes, the best code is no code. The only part of MDAL for which ordered-ness matters is the compiler, which can skip generating output orders for unordered Groups. Also, there are some corner cases that affect the Bintracker GUI. For example, editing should be disabled for the order of a Group containing a single, fixed-length Block member (think fixed length wavetables).
In its current form, sample/wavetable support is rather minimal. There are no graphical editing capabilities yet, Bintracker just shows plain numbers for everything. The plan is to keep extended GUI features such as plotting/drawing samples out of the main application, and implement them as plug-ins instead. I would love to start working on this, but there are some more pressing issues, see below. I might do PCM WAV import though, since we already have a resampler in MDAL. In any case, I modified the wiz4v engine to use user-definable wavetables, so give that a try if you’re already running Bintracker.
Long story short, all of this means that we’re down to one major task that needs to be done before the beta release. That task is implementing support for Label commands. Label commands are needed to implement loop points, for example. I already implemented the GUI part of this, and am now in the middle of tackling the backend part. Now, don’t hit me, but I’m actually going to rewrite the compiler once more. The current implementation has grown somewhat organically, to the point where it has become quite a mess. The compiler solves a complex task, so some complexity in the code is to be expected. However, by now even I have problems understanding my own code, which is a clear sign that I should clean this up before bolting on even more functionality. A rewrite will also allow me to address a few other issues along the way. For example, references are currently resolved immediately. This has two problems. First, it means the compiler cannot resolve forward references. Second, it cannot count references, which is something the optimizer will require. Speaking of which, I’ll probably also implement a simple optimizer while I’m at it. I need to touch all the relevant parts of the code anyway, and I have some ideas as to how it can be done with little extra effort.
Once the compiler rewrite is done and Label commands are implemented, it’s time to prepare for the beta release. I finally got around to replacing my development machine’s semi-broken Gentoo installation with a shiny, up-to-date Void Linux one. This should clear the road towards setting up all the tools that are required for building releases. Nevertheless, I’m expecting this to be rather challenging work. The main obstacle here will be providing builds for Windows. Anybody out there willing to help with this, please get in touch! Just reply to this newsletter, or open an issue on Github if you prefer.
December 18, 2020
Time for a little sign of life! But first of all, my sincere apologies for the prolonged silence. Both the process of getting settled in a new country as well as a number of side projects are taking much bigger chunk of my time than I had originally expected. So, here’s the bad news right away: I have not been working all that much on Bintracker the past months, and things are still not ready for beta release. Of course this also means that the planned crowdfunding campaign will not go ahead just yet. Fortunately that didn’t stop some nice folks from funding development already in the meantime. Special thanks to Denis Perevalov for the big donation!
On the bright side, I did manage to get at least a handful of things done. I completed all the basic functionality for the scm2wiki documentation tool, and released it as an official Chicken extension. The next step will be to integrate scm2wiki into Bintracker’s internal help system.
Secondly, I finally got around to fixing a number of long-standing issues in the edit system. I also implemented an important feature that was long overdue: Selecting things with the mouse. This means the edit system is mostly complete, save for some rough edges.
There’s also one new feature that is perhaps a bit gimmicky, but I’m sure some of you will find it quite useful: Interpolation can now leverage modifier commands. So what does that even mean? Say you want to simulate a slide or glissando on a sound engine that does not natively support these. For a larger interval, you can probably get away with simply interpolating between the start and end note. However, this obviously isn’t going to help when you try to create a slide between two adjacent semitones. Modifier commands to the rescue! When you interpolate a selection that contains both the note column and the associated modifiers, then Bintracker will create a smooth slide by inserting the appropriate detune values into the modifier column. Sounds confusing? Don’t worry, I’ll do a video about this soon.
The MDAL compiler has learned a few new tricks as well. Most importantly, it can now export modules to assembly code. Also, it now supports order lists using relative offsets, which means support for a certain exiting new ZX Spectrum beeper engine is around the corner.
The plugin department has also seen some improvements. Plugins can now extend toolbars, so they can add their own custom buttons to the graphical interface, and modify the behavior of exiting ones. I also began work on a new plugin which will handle export to emulator formats. For now, it only does ZX Spectrum .tap though - with shiny, customizable BASIC loaders, nontheless.
So where do we go from here? Currently I’m taking yet another break from Bintracker development, as my band mate dat zekt and me are busy recording a new Doppelplusungut album. Afterwards, I have another secret little project to work on, and I’m also planning to compete in a couple of music competitions in January. All in all I hope to restart work on Bintracker around mid January.
July 31, 2020
Hello Friends of Bintracker,
Surprise, surprise! Still managed to get a few things done since the last update, even though Bintracker development is officially on hold for the time being. Real life things evolved a bit slower than expected, which gave me the opportunity to work on some smaller features here and there. This will be the last batch for now, though, as I expect to be fully tied up for the month of August.
Without further ado, let me announce a new feature that many of you folks out there will be happy to hear about: MOS 6502 support is done! This means Commodore’s and Atari’s lines of 8-bit computers are within reach, among many others. For now, I’ve added support for the Atari VCS/2600, in form of my tiatune player. I also want to try and add Paul Slocum’s classic Music Kit 2 at some point. However, this is quite low on the priority list, considering there already exists an excellent tool for making music with standard TIA capabilities in the form of TIAtracker.
I’ve also added the Motorola 6809 instruction set, which in turn allowed me to add one of my favorite platforms, the Dragon 32/64 (a clone of the Tandy CoCo, so essentially that one is also supported now). The only supported player for now is The Invisible Man’s Buzzplayer, but there are more to come for this machine’s lovely 6-bit DAC. The 6809 instruction set itself turned into some rather nasty spaghetti code, though. Seems like the assembler’s design isn’t a very good match for such RISC-ish instruction sets. It’s not a huge problem right now, but it’ll inevitably come back to haunt me with AVR8 and 68000, so I definitely need to do some thinking here.
Last but not least, I’ve also added support for the Exidy Sorcerer, a rather obscure Z80-based machine that flopped in the US (Exidy’s home country), but apparently enjoyed some moderate success in Australia, the Netherlands, and Belgium. I had never even heard of that machine, but was recently approached by Exidy enthusiast Michael Borthwick, who told me about his research into the 4 Voice Music System, a rare 6-bit DAC-based sound extension for the machine. Since I always love me some 6-bit sound, I couldn’t resist hacking up a quick and dirty test engine for the thing, so here we are.
Aside from new platforms and engines, I’ve also done some work on the GUI, MDAL, and started to work on some more plugins. Furthermore, I’ve been improving exception handling across all of Bintracker, getting rid of one rather clunky dependency in the process.
For the GUI, I’ve been expanding the core edit system, which is nearing completion. Bintracker now has Porous Paste, Transpose, and various other quality of life improvements for editing modules. I’ve also finally done some work to improve the interaction between the block (pattern) and sequence (order list) display: moving around in the sequence will now highlight the matching block instances, and moving around in blocks will in turn highlight the matching sequence row. Also you can now jump between blocks and orders with the Tab key. The main things that are missing now are folding, support for selecting by mouse, and support for multi-tabbed layouts, that is, display for engines that use tables, samples, and so forth. The display itself works already, but focus control doesn’t, yet (that’s the part of Bintracker that tracks which part of the interface you’re currently interacting with). Furthermore, I did a number of low-level improvements, such as integrating dialog popups with Bintracker’s meta-widget system. This means the plugin API is pretty much stable now, so feel free to start hacking on plugins, if you like!
Unfortunately, there have also been a number of setbacks. I tried to implement tooltips (to replace the information that is displayed in modelines when hovering over buttons), but this destabilized the entire application, so I had to disable them for now. I’ve also still not managed to get panes to scale properly, which in turn is holding back folding. Looks like the underlying ttk::panedwindow implementation is not really made for what I have in mind, so I probably need to replace that with something else.
On the bright side, MDAL now supports modifier commands. Let’s hear it for detune, yay! In preparation of things to come, I’ve also added support for WAV files, including a high quality resampler. The latter was quite the challenge, since I’m only just starting to rediscover my love for maths, after having it ruined many years ago in high school by a string of awful teachers.
Regarding plugins, I’ve started to work on a funky pseudo-random number suite. Implemented some modern generators, as well as a number of historical ones, including von Neumann’s Middle Square Method, and IBM’s infamous RANDU. This work is hardly a priority at this point, but it’s something I can do whenever I’ve got half an hour to spare here or there, which would otherwise not be enough to work on more complex features. In any case, I think all you algoravers out there will find this useful later on! For now, you can already check out the “unzufall” plugin, which is a graphical front-end for the PRNG suite.
That’s all for this time. Hope you all stay healthy, and enjoy the rest of the summer!
June 19, 2020
Hello Friends of Bintracker,
Here’s another update before taking a break from Bintracker development. Things have progressed nicely, managed to get quite a few things done in the past couple of weeks. Among the highlights this time are support for two new platforms, a vastly improved assembler, and several new sound engines.
The most important news is that the major rewrite of Bintracker’s assembler is done. There’s still much room left for optimizations, but considering the lag on note input is overall quite tolerable, I will tackle this at a later point. The code is now much cleaner, no more ugly global state. The main change is that assemblies are now closures. This fulfills an important requirement of libmdal: Support conditional player code that depends on parameters in module data, and do so at least somewhat efficiently. The way it works is roughly this: On loading or creating a module, the MDAL compiler attempts to assemble the player with a limited number of assembler passes. If this succeeds, the compiler caches the result and uses that during module compilation. If it fails, then the compiler caches the closure holding the assembly, and then finalizes the assembly once the missing information becomes available during module compilation. One small tweak I’ve also done is change the output format of the assembler, so you can now run such output directly on an emulator without having to awkwardly convert it first.
I’ve added support for one new instruction set (Fairchild F8), and two new target platforms: The Fairchild Channel F console, and the Sharp MZ-700 computer. The former was surprisingly hard to do. I hadn’t anticipated that hot-reloading on cartridge based systems is very different from doing it on a regular home computer core in MAME. Either way, I managed to pull it off in the end, and with that we now (hopefully) have a workable blue print for other consoles. Adding MZ-700 on the other hand was a bit disappointing. Apparently sound support for this machine in MAME is not very accurate, so its usefulness remains limited for now.
A number new engines have been added: SleizsaTrio for the Channel F, Squeeker and PhaserX for the ZX Spectrum 48K, as well as MooZ’ PhaserX port for Sharp MZ-700. As mentioned above, the latter doesn’t sound all that great in MAME though. You can expect the number of engines to grow much faster after the scheduled development break: As libmdal and the assembler get more complete, it becomes easier and easier to add more of them.
Much to my frustration, I hadn’t been able to include a working plugin in the initial source release. I found the obnoxious bug that was preventing my test plugin from working a couple of days afterwards, so we now have our first plugin! It’s nothing too impressive, just a simple Music Macro Language parser, so you can now write melodies in MML.
Speaking of bugs, I’ve been fixing those left and right. At this point I’m nearly done working through the big stack of notes on known bugs that I was holding up in the last video. What’s left are mostly UI-related bugs, which are mostly nuisances rather than critical crash-causing errors. There are two exceptions here, which I will touch on in the next section.
Furthermore I’ve completed the big MDAL rename I had planned for a while. So, no more “MDAL Configurations”, they’re now called “MDAL Engine Definitions” or short “MDEF”, which I believe makes their purpose much clearer. The libmdal API is now also significantly more consistent, though there’s still a fair bit of work left to do in that respect.
Last but not least, I’ve done a number of small improvements to the UI, as well as improving discoverability. For example, moving around in the module now displays more detailed information of what’s under cursor, and the “New Module” dialog now displays information on the selected engine.
Normally this where I explain what I’ll do next. However, this time I’ll give a more general overview of what still needs to be done, a sort of general road map, if you will.
As mentioned, there are a couple of known bugs left, and taking care of those is the top item on my to-do list (though I’m not of course not always in the mood of fixing bugs, implementing new features is much more fun!). One major bug that’s still alive is related to Undo/Redo: Registering block edit actions (copy/paste etc.) in the edit journal is not fully implemented yet, so calling Undo after such an action may break things in spectacular ways. The other bug is related to emulation: Row play will sometimes fail, producing horrible noise instead. This is caused by a certain class of bugs in MAME’s CPU cores. The MAME team is aware of the problem, but since the issue doesn’t really affect “normal” users, I think fixing this has pretty low priority for them (and I don’t blame them, in fact they’ve been pretty helpful as of late). So I’ll need to find a work-around for this. There is also some dodgy behavior going on in popup dialogues. This will probably require some digging into pstk, the Chicken Scheme extension that provides the interface to the Tk GUI toolkit.
Another item I want to get started on soon is building Bintracker on Windows. My aim is to provide a self-contained, statically compiled package that’s as easy as possible to set up. I haven’t used Windows since the XP days, so surely this is going to be… fun. Any help with this is highly welcome!
Furthermore, I’ll be adding support for more CPU architectures and target platforms. Next on my list are the Atari VCS/2600, and Tandy CoCo/Dragon 32/64, which means adding the 65xx and 6809 architectures to the assembler. Once 65xx support is done, adding the Commodore line of home computers will probably be the next step.
Documentation is still sorely lacking. After going on a writing spree before the source code release, I’m pretty burned out in this regard, but I’ll get back to it eventually. Also, I still need to make adjustments to scm2wiki, the tool that generates the API documentation. I actually want to integrate scm2wiki into Bintracker itself, to make it self-documenting. During the last round of work on scm2wiki, I already implemented the necessary changes to make this possible, but nothing is done yet on the Bintracker side.
There are a number of crucial features still missing in both the Bintracker GUI and the MDAL compiler, which amount to a sizable chunk of work. These form the bulk of what needs to be done before the beta release. I estimate that I will need around 8-10 weeks to complete those. MDAL misses three major things at this point: Support for unordered blocks (which will be used to implement things like samples and effects tables), modifier commands (which will be used to implement detune), and the output optimizer. I already have a pretty solid concept for the optimizer. However, I need to do unordered blocks first, and I’m still very much in the “throwing around ideas” phase regarding that. Two smaller things that are also still missing is loop points (still waiting for a eureka moment here as well), and overrides, which will offer a way to tweak the engine definition used by a module. The latter will be the basis for implementing microtonal scales and alternative tunings.
Regarding the GUI, there are a number of things that still need to be done. One of the most heavy duty jobs here is implementing simplified sequences and automatic sequences, though the latter will very likely have to wait until after the beta release. Another big item is folding. See those double-arrow buttons on the left side? The idea is that you can click those to collapse the buffer that they’re attached to. It’s not just a convenience feature: Sound engines with a more complex data structure (think module -> subtune -> sequence -> chain -> patterns -> instruments -> tables) will definitely require this. Aside from these big baddies, there are plenty of smaller things left to do as well, especially in regards to cleaning up the user interface and improving/extending the plugin API. There are also a couple of MDAL command types that aren’t fully supported yet.
Once all these things are done, the core of Bintracker will be mostly complete. Which means I can finally focus on adding more sound engines, and even more exiting: Plugins! Already got a huge list of ideas for plugins, and probably I won’t be able to resist the urge of implementing some of those before finishing the core features mentioned above.
I also want to make a more active effort to build a community around Bintracker. At this point I feel that I’ve implemented the core parts of my vision of what I want Bintracker to be, so it’s time to move from “me” to “we”, so to speak. Needless to say, I’m counting on the support of all you good folks out there, which brings me to…
Since I squashed most of the bugs I knew about, I’m now more than happy to receive user testing and bug reports! There’s definitely plenty of bugs left, I’m just not aware of them.
The main topic on which I would very much appreciate some input and help concerns infrastructure. There are a number of things that come to mind. One thing I would like to have are synchronized mirrors of the main Github repository. I’ll probably set up a mirror on Gitlab myself, but if some of you were to self-host additional mirrors, that would be great.
Also, I’m wondering what you folks would recommend as a bug tracker or project management platform in general. I guess for now we’ll stick to Github Issues, but in the long term I would prefer a solution outside of Github, to avoid becoming vendor-locked. However, I don’t want to self-host this on the Bintracker website. The same goes for a project wiki. Let me know your thoughts on that.
Another issue concerns setting up continuous integration, ideally with automated nightly builds. Obviously this will require figuring out how to build on MacOS and Windows, first. I will probably try to tackle Windows myself soon-ish, but as mentioned previously, I would very much appreciate help with MacOS.
Furthermore, I would like to hear your opinions on communication infrastructure, ie. setting up a chat and/or forum. Do people even use forums in 2020? Do we need one? I mean I like them, but as things stand now forums seem to be in decline everywhere. Regarding chat, I personally favor IRC (yeah I’m old), but I’m open to suggestions. Not a big fan of Discord, but I understand that’s where a lot of people hang out nowadays. So perhaps an IRC channel bridged to Discord would be a good option. Would be great if someone trustworthy would step forward to take care of this, as I don’t have any experience with it myself, and my internet access is often spotty so I can’t run a bot. As an alternative, I’ve been looking into Gitter. I like how it integrates with Github/lab repositories, but again I’m kind of weary of vendor lock-in.
Lastly, please keep spreading the word about Bintracker! The more people get interested in this, the more motivated I will be. Perhaps it’s also time for establishing a presence on Twitter/Mastodon and Facebook. I would very much prefer not having to do that myself, so if anybody out there wants to take on the job, by all means go for it (talk to me first, though!).
Let me just say that overall I’m pretty happy with the recent progress. Compared with the initial source release, stability has improved a lot, and Bintracker is slowly shaping to become a tool that’s actually useful, rather than just an experimental mess. That said, there is obviously still a ton of work left, which I’ll start to tackle in about 2 months time, after taking a break to take care of some big personal life changes. I might be able to get some smaller things done during this, but I can’t promise anything because it’s hard to say how packed my schedule is will be in the coming weeks. So for now, stay tuned, and as always, let me know about your thoughts, questions, and suggestions. Note that I may lose internet access for a couple of weeks in late July/early August, so don’t be mad at me if I don’t reply to your messages immediately.
May 30, 2020
Hello Friends of Bintracker,
Time has come! I’m releasing the Bintracker source code today. Now, before you get too exited: Bintracker is definitely not ready for production use yet. That said, most of the essential features are there, and if you’re adventurous enough and know how to compile from source, you can definitely start to some fun with Bintracker now.
All the things that I talked about in the previous video update are done. I rewrote the build system, so Bintracker now neatly builds out of source. Copy/paste (mostly) works. There is a second engine (nothing too exiting, just good old Quattropic), and documentation is coming along nicely (though still far from being complete). Also, I have added the plugin API. I’ve started to work on some plugins too (MML support coming up), but haven’t managed to complete any of them yet.
There are a number of bugs in the edit system, I will start tackling those in the coming days. For the time being, remember to save often when you make music with Bintracker.
The next big step will be reworking the assembler, and adding new instruction sets to it, which in turn means more supported platforms, which in turn means more engines. Also, I will continue to work on plugins, as well as extending the MDAL compiler. Furthermore, I want to get started with Windows builds.
That said, I will now continue to work on Bintracker for a few more days, but then I will take a break for about 2 months because I’ll have some “real life” things to take care of. I’m hoping to start up development again by the end of summer.
I guess this is a good moment to say some thanks. Thanks to all of you who have been following the project until now. Massive thanks to my girlfriend, who has been supporting me throughout this project. Special thanks go out to the CHICKEN Team, especially Felix Winkelmann, who has been giving me a lot of advice and help. Thanks to adkd, nyanpasu64, and Alone Coder, the discussions we had really helped move things forward. Thanks to Shiru, whose is to blame for the original idea of MDAL. And a big thank you to Catskull Electronics for the big donation.
Speaking of donations, the crowdfunding campaign will of course be delayed until the first beta release is out, which will be in late autumn, as things stand now. If you do feel like sending some financial support in the meantime, I’ve set up a Paypal donation button on the website for now.
That’s all for now. As always, let me know if you have any suggestions/questions/etc, and of course check out Bintracker!
March 06, 2020
After a very busy December and first half of January, Bintracker development is now in full swing again for the time being. Unfortunately there are some bad news. Due to some unforeseen circumstances, I will have much less time for Bintracker in the coming months than I had originally hoped. In other words: I’m afraid I won’t be able to keep my promise of releasing the first beta by the end of Spring. I will try my best though to at least release the source code by then, so you all you hackers out there have something to play with over the summer.
The pattern/sequence display rework mentioned in the previous newsletter is complete. The new implementation is based on a single Tk Text widget, rather than a hack to combine several Treeview widgets. This reduces the complexity a fair bit. More importantly, it also results in a significant speed boost, so the display is much more responsive now. It’s also more compact, making better use of vertical and especially horizontal space. Check it out:
I also made good progress on rewriting scm2wiki, my in-source documentation tool. Before the next public release I still need to add back the svnwiki backend (not used for Bintracker itself, but a couple of other projects of mine). As far as Bintracker and libmdal are concerned, scm2wiki is now capable of creating proper documentation for the entire API. Also, the code is now nicely modular, which means I can potentially use it to generate documentation that can be used in Bintracker itself, so Bintracker would become self-documenting.
I’ve moved on to the MDAL compiler again. The most important news here is that the old, custom MDMOD format is no more. MDAL modules are now symbolic expressions, which brings them in line with everything else in Bintracker. For those of you who want to know all the nitty gritty details, check out the latest MDMOD Specification Draft. Gotta love the new MDMOD parser: It’s a whopping 125 lines of code, including a major upgrade to the error handling.
While I’m at it, I thought it might be a good idea to change the internal representation of MDAL modules in libmdal, too. Yak shaving commences… I think it was very much worth spending a few days on this, though. In fact I just got the compiler generator to work again with the new design, just need to fix up a few loose ends here and there now. The original implementation had some heavy overhead in places, due to using custom datatypes to represent the input nodes of a module. The new design uses only alists (aka associative lists, a common and simple way to implement lookup tables in Scheme). Block fields nodes (which are used to represent channels in a pattern, for example) are abstracted away completely, so instances of Block nodes now contain just the raw data, organized in rows. I’m not entirely sure yet if the latter was a smart decision, but I’m rolling with it for now. In any case, all those changes brought libmdal’s code size down by a few hundred lines, so I think we’re on the right track.
Furthermore I had a great idea that will be helpful in tackling various problems regarding sequences. The idea is to simply make Block nodes endless by default, and control the actual length through an explicit length parameter in the sequence. So what exactly does this solve? Say we have a set of patterns (per-channel, since MDAL uses matrix sequences as the lowest common denominator to represent all sequence types found in existing sound engines), and we want to use them at a given step in the sequence. What happens if the patterns aren’t the same length? Do we pad shorter patterns up to the length of the longest one, or do we shorten everything to the shortest one? If we opt for the former, how do we even represent this in the GUI? What does it mean to “insert a row” in such a setup? Endless Block nodes basically allow us to bypass all these questions. No block length mismatches can exist, and adding a row to a block does not change the length of any step in the sequence. And the best part is: The new internal module representation makes this very easy to implement, and in fact I have already implemented the basics of it.
Once I’ve finished the next iteration of the MDAL compiler, I plan to finally tackle the emulation part, ie. connecting Bintracker to MAME. I’ve got a reasonably worked out plan on how to do things on the MAME side, but I still don’t have a clear picture how to implement things on the Bintracker side. Collecting some ideas at the moment. In terms of concurrency, CHICKEN offers a standard SRFI-18 implementation. The alternative would be gochan, which, as the name suggests, is inspired by Go Channels.
Implementing emulation is the major obstacle on the way to the first release of Bintracker. However, there are a number of smaller items on the to-do list as well. These include:
Several people got in touch with questions and suggestions after the last newsletter. Very happy about that, please keep them coming. Special thanks to adkd and nyanpasu64 (who’s working on a next-gen NES tracker) for the great discussions.
I prefer a compact user interace, like Protracker 2.3d, with a full set of keyboard shortcuts. Will this be possible with Bintracker?
Yes, this is in fact a major design goal. There will be keyboard shortcuts for everything, and they will be customizable, too. Regarding compact UI, you can disable toolbars and the menu (and of course the console) to increase vertical space. Font size can also be changed. Also, the re-worked main module view is more compact horizontally, and will have configurable line spacing as well. Furthermore, I want to make channels “collapsible”. So in collapsed mode, each channel will only show the primary column (ie, note column or trigger column for drum channels).
I would like a “search box”, for things like finding uses of a certain instrument in the song, or searching for a certain element by name.
That’s a great idea. It would be a great showcase for a Bintracker plugin. I’m actually liking the idea to the point where I’m considering implementing it for the first beta release.
Are you intending to write a custom audio renderer for Bintracker?
No, at least not for now. For the moment, the plan is to just let MAME do the audio. In the future Bintracker will support multiple emulation backends. For each backend, there must exist an abstraction layer which exposes at least a minimum set of features (eg. run/stop/load), and may optionally support more advanced features like low latency playback or MIDI IO.
I do have an idea for adding a sort of native emulator at some point. It would be in the form of an abstract, customizable virtual machine running low-level bytecode. MDAL configurations could then include a bytecode translation of the sound engine which can run in the VM. The main use of this would be to provide support for machines for which no viable emulator exists, such as mini-computers and mainframes. Either way, this certainly won’t happen any time soon.
What’s the plan regarding Undo?
MDAL uses a tree to represent module data. (I considered using a dictionary instead, but it would be too impractical considering MDMOD allows for infinite recursion.) All changes to the tree pass through a single dispatch procedure. The dispatch procedure invokes one of 4 edit procedures, depending on the argument list (set, insert, remove, or compound-edit). It returns an argument list. Calling the dispatch function with that argument list reverses the initial edit. The returned argument lists are managed by two stacks for undo/redo. In other words, the dispatch function is handed a description of the change, mutates the document, and returns an undo description.
Will Bintracker support macros, and if so, what’s your plan regarding security?
Yes, MDMOD will support macros, in the sense that any data can be substituted with Scheme code. Regarding security, Bintracker is basically a Scheme interpreter, which is exposed to the user at runtime. In this sense it is insecure by design. However, this is a conscious decision: My idea is for Bintracker to be this hacky tool that you can use for pretty much anything. So you should be able to use Bintracker like a tracker, but also like an IDE or a live programming environment, for example. That said, I’m of course considering mitigations especially in regards to plugins. The general idea right now is to have an “official” central repository which only admits plugins that have been vetted, and display a warning when users attempt to install a plugin from an untrusted source. For mdmod files, I may actually sandbox the interpreter, though this poses some challenges with regards to file access (externally stored sample data etc).
I’ve had multiple discussions on the subject of sequences that I touched upon in the previous newsletter. I’ll do a separate write-up on that in one of the upcoming newsletters. In the meantime, if you have any more ideas on that, I’d be happy to hear them, of course.
Another subject I’d like to get your input is “folding”, as in “code folding”. For those of you who aren’t coders, code folding is a feature in some programming editors that allows you to collapse parts of the code that aren’t currently relevant to you. For example, say you have a loop, then folding the loop would hide anything but the first line of the loop code from view. So programmers know that there is a loop there, but don’t need to scroll through the details of it unless they actually want to work on that. I believe that this concept could be useful in a tracker as well. However, I haven’t gotten any further than this vague idea in my head. So if anyone has some thoughts on that, let me know!
Pretty much all the points from the last newsletter still apply. Some of you good folks out there are already spreading the word on social media, so thanks for that!
Still looking for someone to help with custom icons, and with MacOS builds and/or CI things in general. I also could do with some advice on how to securely implement an update function that pulls the latest plugin version from the planned central repository.
Well, I guess that’s all for this time. I’m planning on sending out newsletters a bit more frequently from now on, but with a twist. Instead of giving a full update each time (which will still happen of course), I’ll mix things up with write-ups on one specific topic at a time. The plan is to eventually make these write-ups part of the Bintracker documentation as well.
As always, if you have any questions, suggestions, ideas, or you want to help with stuff, please get in touch.
December 12, 2019
Here’s what I’ve got so far.
Most importantly, MDAL, the back-end library that powers Bintracker, is working. MDAL does 3 things.
As mentioned in my talks, MDAL is Turing-complete, which means it can compile MDMODs to any target data format. In other words, MDAL can already compile modules for any chiptune music engine out there, though more work is needed to make it convenient to do so. By the way, MDAL can also run as a stand-alone application. If you’re curious, there is some in-depth (if somewhat out of date) information available on utz82.github.io/MDAL.
In Bintracker itself, a number of components are already working. This includes the built-in console, which famously allows you to program Bintracker in Bintracker. While this is a feature that probably not everyone will use, it’s proven to be very helpful with debugging during Bintracker development. Furthermore, the user configuration system is up and running, so colors, fonts, and most key bindings are already user-configurable. In fact the configuration system is extremely powerful: By adding code snippets to your configuration file, you can tweak virtually any aspect of Bintracker.
For the past weeks, I’ve been working on designing and building the main interface. Been making steady progress, and it’s starting to look pretty slick if you ask me (see front page).
As you can see, things are certainly more colorful than what I showed in my talks in August and October. Bintracker now has actual color themes, and there’s already several ones to chose from. Yeah, sometimes I can’t force myself to work on the actually important parts, so I implement stuff like this instead. ;)
A major part of the work on the GUI involved a rework of the pattern and sequence display. With the first, naive implementation, Bintracker could only display one pattern at a time. The new pattern display is continuous, meaning you can scroll through the whole song without leaving the pattern display. Pulling this off was quite a bit more complicated and time-consuming than I had hoped.
Unfortunately there’s some bad news here: I realized there’s a fatal flaw in the current pattern/sequence display implementation. Say you have a two-digit volume column, and you want to set the volume on all rows to 40. In a good tracker, you’d move the cursor to the start of the pattern, and position it on the left digit of the volume column. Then you’d hold down key 4 to fill the whole column with 40s. With Bintracker’s current design, you can’t do that, because it isn’t possible to position the cursor on a single digit of a numeric column. In essence, this means re-doing some 500+ lines of code. Already made a good start on it this week though, so I’m confident I can finish this before the end of the month.
On the bright side, I also made a start on the edit system. While it’s far from complete, you can already perform simple edits like adding/deleting notes. More importantly, the undo/redo system is nearly complete. Really happy with it, as it’s super minimalist (less than 100 lines of code), yet very robust.
Speaking of lines of code, the project recently passed the 5000 line mark. About half of those make up the MDAL library, and the other half is Bintracker proper. While I’m not exactly proud of this “milestone”, I think it’s reasonable for what Bintracker can do at this point. I do a bit of refactoring to reduce the code size once in a while, however there should still be room for improvement. In any case, I hope to keep the Bintracker core below 10k lines of code for the initial beta release.
Once I’ve crossed the aforementioned pattern/sequence display rework and the edit system off the list, the next 3 major items will be the plugin API, the emulation back-end, and enhancements for the MDAL compiler.
While the plugin API isn’t really a requirement yet at this point, I’ve already got a detailed plan for the implementation on paper. So I basically just need to type up the code and write some support functions. Having a few example plugins can also help with testing the edit system beyond the standard functionality provided by core Bintracker. Once the basic plugin API is implemented, it’s also time to start thinking about plugin management. However, this will have to wait until after the first beta release.
The next big step is the emulation back-end. As you may recall, the plan is to default to MAME for emulation. MAME has a built-in LUA console, which allows you to script a running MAME instance. So essentially I have to get Bintracker and the MAME console to talk to each other. I’ve done some preliminary tests for this, and managed to get Bintracker to talk to MAME through a pipe (much like Bintracker and Tk, the GUI widget toolkit communicate with each other). However, in order for this to be useful I will need to implement multi threading on both sides, which is an area that I’m not exactly competent in. Well, I’m sure I’ll be learning a lot.
Once emulation is working, I’ll move on to work on the MDAL compiler again. Needless to say that this will be very dry and challenging work. There are a number of things to do here. Most importantly, I need to add several missing features that are defined in the MDCONF standard, and add range of different abstractions to make writing MDCONF files more convenient, so people can actually write their own configurations without going insane. Then, I need to add more instruction sets to Schemta, MDAL’s built-in assembler. Right now Schemta only supports the Z80 CPU. For the first beta release, I want to at least add support for the MOS 6502 and derivatives, Motorola 6809, and Fairchild F8. Last but not least, I want to change the MDMOD standard to a symbolic expression based format. Right now MDMOD is a custom format, which is a hold-over from the very early days of MDAL. The current MDMOD parser is one of the most beautiful things in the whole Bintracker code, and I’m kind of sad to see it go. However, using symbolic expressions will make the MDMOD format much more flexible and extensible, which will save me a lot of work in the long run. It definitely payed off when I changed to MDCONF format a few months ago. And all that aside, MDMOD is actually the last part of Bintracker that doesn’t use symbolic expressions.
There are also many different small things to do before the first beta release: Tweaking and fine-tuning the GUI, expanding the edit system, writing the user manual, coding some example plugins, writing exporters for a range of emulator formats, etc. My custom in-source documentation tool also needs a major upgrade. I hope to do that during my trip to Germany in next month.
Speaking of which, I’m going to take a break from Bintracker development in December. I’ll be in Germany for most of the month, recording a new trash music album with by band Doppelplusungut. A note if you want to get in touch: Our studio is located in the middle of nowhere, and going by previous experience internet access can be rather wonky out there. This means that I may not respond to messages as quickly as I normally would during that time.
Several people have already offered to help with testing Bintracker, writing demo tracks, and so forth. Please know that those offers are very much appreciated! Unfortunately, we’re not quite there yet - it will be a few more months till Bintracker is in a usable state. However, there are a few other things that I could use help with at this point.
At the moment Bintracker uses Material Design icons, but there’s a few tracker specific things that are missing from that set. For example, of course Material includes buttons for Copy/Cut/Paste and so on. However, in a tracker, there are for example two different ways to Cut. You can either cut a section and leave the following section in place, or you can cut a section and shift up whatever comes after it. Since we want a toolbar button for each of those, this of course requires two different button icons. Paste is even worse. Bintracker supports 4 different paste operations:
Other icon sets I’ve looked at (Feather, FontAwesome, IonIcons, Linea, OpenIconic) won’t be of any help either, if anything they’re even less complete. So there are basically two options: Either derive the missing icons from existing Material Design ones, or design a completely new, custom set of icons. In any case, I’m not the most talented graphician, so I’d really appreciate if someone could help out with this!
Any developers out there who would be willing to help with building Bintracker on Mac? I’m not a Mac owner, so I have very little experience with OS X myself. Building Bintracker itself shouldn’t be too terribly hard. However I’d want to provide a convenient, easy-to-install package to users that will download and set up the necessary 3rd party software (MAME and Tcl/Tk) as well. As far as I know, Tcl/Tk is installed by default up to Mojave, but Apple stopped shipping it in Catalina. Also, ideally I’d want a CI pipeline for automatically building and deploying new releases.
You probably guessed it: I’m not a big fan of social media, and in all honesty I’d much rather spend my time working on Bintracker itself, than promoting it. That said, for Bintracker to become a success, it will be helpful to spread the word as much as possible, especially in regards to crowdfunding (more on that below). So, any volunteers for running a Twitter account and/or spreading the news on Facebook etc.? Please get in touch.
I currently have the privilege to work on Bintracker more or less full-time. However, the good times are coming to an end soon. That means I will either need to take up a regular day job (in which case Bintracker development would obviously slow down significantly), or I need to find another way to generate some income. My initial idea was to run a Kickstarter, but I’m anything but sure that this is a good idea.
The problem with running a Kickstarter is that it can of course fail, in which case I will have put in a lot of work for nothing. And considering that I’d need to raise a quite substantial amount (mind you, a sizeable chunk will go to taxes and Kickstarter fees), plus my sub-par promotion skills, the chance of failure is quite high. I know that I can run a flex-target campaign on Indiegogo, but that would incur even higher fees. Plus, Indiegogo has a pretty bad reputation in some parts of the retro scene. Alternatives? GoFundMe doesn’t accept Paypal, which would put off quite a few people, I’m afraid.
A possible alternative would be to set up a Patreon account. This would carry a much lower risk. However, clearly not everyone who would generally be willing to throw some money my way, would necessarily be willing to do so every month. So if I put my bets on Patreon, I will definitely lose out on some potential income.
So why not try both? Well, then people who opt for Patreon won’t put money into the Kickstarter, increasing the chance of failure even more. And vice versa, people who put money into the Kickstarter won’t feel inclined to put money into the Patreon. So this is a lose-lose situation.
Not sure what’s the best course of action regarding all this. If you have any ideas, please let me hear them. Also, if you would be willing to help with running a crowdfunding campaign and/or setting up other donation options, please get in touch.
Aside from crowdfunding/donations, I’m also open to other funding options. Anybody out there who knows someone at a company that might be interested in becoming a sponsor for Bintracker? With Bintracker being free and open source, there isn’t much I can offer in return, but I would be more than happy to put some sponsor’s logos up on the Bintracker website. I’m also open to the idea of a company hiring me to work on Bintracker. Bintracker could be branded “/YourCompanyNameHere/ Bintracker” in this case. Hiring could potentially be done through a foundation, so any money spent would be tax-deducible, or if the company is doing audio related products, they could hire me directly, allowing me to spend a part of my working hours on Bintracker.
Last but not least, I would also like to get your feedback on the newsletter itself. Is the length OK? Would you prefer shorter, more frequent updates? Is there anything missing? Also, if you have any questions or suggestions for Bintracker, feel free to send them in and I’ll discuss them in the next newsletter.
There are a couple of specific topics that I would like to get your opinion on.
OK, in order to explain this, I’ll need to ramble a bit. Bear with me, or just skip this section if you don’t care.
In my opinion, sequences (aka order lists) are a weak point of trackers in general. The main gripe I have with them is that they interrupt your flow of thought when you’re composing. Also, the fact that there’s no common standard for them in existing trackers seems to underline the idea that there’s something wrong with them. There are two common approaches: Matrix and Unified sequences. By matrix sequences, I mean those where each channel gets its own track in the sequence (like in Famitracker), whereas XM trackers are an example of unified sequences - there is only one track in the sequence, which sets all the channels.
Personally, I’m a big fan of 1tracker’s approach, which ditches sequences completely, and instead uses “sections”. To start a new “pattern” (or rather section) in 1tracker, you just hit the space bar in the “endless” pattern roll. Hit space again to merge back with the following section. This does mean you’ll do a bit more copy&pasting than usual, but overall I think it’s a net win. Copy&Paste requires maybe one or two more key strokes than inserting a new row into the sequence, but this is more than compensated by the fact that I can just compose my melodies in one go, without interrupting whatever I’m doing to edit the sequence. However, that’s just me of course - your mileage may vary.
Now, matrix sequences make sense when the tracker runs natively on the target hardware, because they allow the composer to optimize for size. However, on a PC, optimizing is a job that the computer can do, so there isn’t any good reason to burden the user with that. Either way, here’s the catch: MDAL (Bintracker’s compiler) uses matrix sequences by default, unless specified otherwise by the sound driver configuration. However, those matrix sequences are virtual, in the sense that the compiler rebuilds all sequences during compilation. There are two reasons for this. For one, it makes optimization easier. Secondly, it’s necessary in order to support sound drivers that require a fixed pattern length (like Huby on ZX Spectrum, for example), without bothering the user with such constraints. More importantly though, this opens up some interesting possibilities for Bintracker itself. Essentially, Bintracker could offer the user a choice multiple sequence styles. Want to fiddle with the low-level matrix? Be my guest. Want a simple unified sequence? No problem, MDAL will just optimize away any redundancies that this introduces. Want automatic sequences like in 1tracker? That’s possible, too.
Ok, so I have some questions regarding this. What is your preferred sequence style? Do you actually enjoy working with matrix sequences, or do you find them annoying? What would be your “dream” sequence style? Also, someone at Deadline (sorry, can’t remember your name) suggested to have “blocky” matrix sequences as an option, like in Jeskola Buzz and most “modern” DAWs, where each pattern is represented as a block of a specific size that correlates with its length. I think that’s a great idea. However, if I implement that, I will drop support for classical matrix sequences, as supporting both would mean I’d have to write a lot of extra code. So, let me know what you think - Blocky sequences: yay or nay?
You probably know by now that Bintracker will be heavy on the plugins (and I very much hope I don’t have to write them all myself :D). Anyway, there are two ideas here: first, plugins must be composable, that means plugins can depend on other plugins. Say we have a low-level plugin for version control, let’s call it “git”. Then another plugin, let’s call it “project manager”, could use “git” under the hood to its thing. And yet another plugin, let’s call it “online collaborator” (yeah, being really creative with names here) could in turn extend “project manager”’s capabilities. I’ve got a pretty solid plan for implementing this already, so that part is more or less covered.
What I’m far less sure about is the second idea. Since there are (hopefully) going to be quite a few plugins, Bintracker will need a plugin manager at some point. The plugin manager should resolve dependencies (eg. when the user installs a plugin, the plugin manager should also install the dependencies of that plugin), and it should also do updates and such. Generally I’m thinking to set up a central plugin repository, and the plugin manager should talk to that repository somehow. However, manual plugin installation should also be possible. I’d actually prefer the plugin manager itself to be a plugin (so it could be replaced with alternative plugin managers), though I’m totally not sure whether that’s a good idea. Also, really not sure how to implement the whole thing. First of all, how to make it secure? Hashes? Signed keys? SSL? I don’t have much experience with crypto, so any help or at least some pointers in the right direction would be much appreciated. Also, where should the central repository reside? Bintracker.org would be the most sensible option in my opinion, but that currently runs on a shared web space, so can’t run git on that. As you might know I don’t have a lot of money, so users would have to pitch in for hosting costs, if we were to switch to a real server instead. The other option would be to have the central repository on a service like Github or Gitlab (possibly with a few fallback options). This would simplify things a lot, as many people are already familiar with the workflow of these sites. However, I’m a bit worried about this thing called “vendor lock-in”… Anyway, I’d be glad to hear your thoughts on this.
Well, that pretty much wraps up this newsletter. I was going to include another section with some insight into the technology and implementation details of Bintracker, for the coders among you. However, this newsletter is getting quite long already, so this will have to wait until next time. Once again, if you any questions, suggestions, ideas, or you want to help with stuff - please get in touch.