BINTRACKER
12 Dec 2019
Here’s what I’ve got so far.
Most importantly, MDAL, the back-end library that powers Bintracker, is working. MDAL does 3 things.
As mentioned in my talks, MDAL is Turing-complete, which means it can compile MDMODs to any target data format. In other words, MDAL can already compile modules for any chiptune music engine out there, though more work is needed to make it convenient to do so. By the way, MDAL can also run as a stand-alone application. If you’re curious, there is some in-depth (if somewhat out of date) information available on utz82.github.io/MDAL.
In Bintracker itself, a number of components are already working. This includes the built-in console, which famously allows you to program Bintracker in Bintracker. While this is a feature that probably not everyone will use, it’s proven to be very helpful with debugging during Bintracker development. Furthermore, the user configuration system is up and running, so colors, fonts, and most key bindings are already user-configurable. In fact the configuration system is extremely powerful: By adding code snippets to your configuration file, you can tweak virtually any aspect of Bintracker.
For the past weeks, I’ve been working on designing and building the main interface. Been making steady progress, and it’s starting to look pretty slick if you ask me (see front page).
As you can see, things are certainly more colorful than what I showed in my talks in August and October. Bintracker now has actual color themes, and there’s already several ones to chose from. Yeah, sometimes I can’t force myself to work on the actually important parts, so I implement stuff like this instead. ;)
A major part of the work on the GUI involved a rework of the pattern and sequence display. With the first, naive implementation, Bintracker could only display one pattern at a time. The new pattern display is continuous, meaning you can scroll through the whole song without leaving the pattern display. Pulling this off was quite a bit more complicated and time-consuming than I had hoped.
Unfortunately there’s some bad news here: I realized there’s a fatal flaw in the current pattern/sequence display implementation. Say you have a two-digit volume column, and you want to set the volume on all rows to 40. In a good tracker, you’d move the cursor to the start of the pattern, and position it on the left digit of the volume column. Then you’d hold down key 4 to fill the whole column with 40s. With Bintracker’s current design, you can’t do that, because it isn’t possible to position the cursor on a single digit of a numeric column. In essence, this means re-doing some 500+ lines of code. Already made a good start on it this week though, so I’m confident I can finish this before the end of the month.
On the bright side, I also made a start on the edit system. While it’s far from complete, you can already perform simple edits like adding/deleting notes. More importantly, the undo/redo system is nearly complete. Really happy with it, as it’s super minimalist (less than 100 lines of code), yet very robust.
Speaking of lines of code, the project recently passed the 5000 line mark. About half of those make up the MDAL library, and the other half is Bintracker proper. While I’m not exactly proud of this “milestone”, I think it’s reasonable for what Bintracker can do at this point. I do a bit of refactoring to reduce the code size once in a while, however there should still be room for improvement. In any case, I hope to keep the Bintracker core below 10k lines of code for the initial beta release.
Once I’ve crossed the aforementioned pattern/sequence display rework and the edit system off the list, the next 3 major items will be the plugin API, the emulation back-end, and enhancements for the MDAL compiler.
While the plugin API isn’t really a requirement yet at this point, I’ve already got a detailed plan for the implementation on paper. So I basically just need to type up the code and write some support functions. Having a few example plugins can also help with testing the edit system beyond the standard functionality provided by core Bintracker. Once the basic plugin API is implemented, it’s also time to start thinking about plugin management. However, this will have to wait until after the first beta release.
The next big step is the emulation back-end. As you may recall, the plan is to default to MAME for emulation. MAME has a built-in LUA console, which allows you to script a running MAME instance. So essentially I have to get Bintracker and the MAME console to talk to each other. I’ve done some preliminary tests for this, and managed to get Bintracker to talk to MAME through a pipe (much like Bintracker and Tk, the GUI widget toolkit communicate with each other). However, in order for this to be useful I will need to implement multi threading on both sides, which is an area that I’m not exactly competent in. Well, I’m sure I’ll be learning a lot.
Once emulation is working, I’ll move on to work on the MDAL compiler again. Needless to say that this will be very dry and challenging work. There are a number of things to do here. Most importantly, I need to add several missing features that are defined in the MDCONF standard, and add range of different abstractions to make writing MDCONF files more convenient, so people can actually write their own configurations without going insane. Then, I need to add more instruction sets to Schemta, MDAL’s built-in assembler. Right now Schemta only supports the Z80 CPU. For the first beta release, I want to at least add support for the MOS 6502 and derivatives, Motorola 6809, and Fairchild F8. Last but not least, I want to change the MDMOD standard to a symbolic expression based format. Right now MDMOD is a custom format, which is a hold-over from the very early days of MDAL. The current MDMOD parser is one of the most beautiful things in the whole Bintracker code, and I’m kind of sad to see it go. However, using symbolic expressions will make the MDMOD format much more flexible and extensible, which will save me a lot of work in the long run. It definitely payed off when I changed to MDCONF format a few months ago. And all that aside, MDMOD is actually the last part of Bintracker that doesn’t use symbolic expressions.
There are also many different small things to do before the first beta release: Tweaking and fine-tuning the GUI, expanding the edit system, writing the user manual, coding some example plugins, writing exporters for a range of emulator formats, etc. My custom in-source documentation tool also needs a major upgrade. I hope to do that during my trip to Germany in next month.
Speaking of which, I’m going to take a break from Bintracker development in December. I’ll be in Germany for most of the month, recording a new trash music album with by band Doppelplusungut. A note if you want to get in touch: Our studio is located in the middle of nowhere, and going by previous experience internet access can be rather wonky out there. This means that I may not respond to messages as quickly as I normally would during that time.
Several people have already offered to help with testing Bintracker, writing demo tracks, and so forth. Please know that those offers are very much appreciated! Unfortunately, we’re not quite there yet - it will be a few more months till Bintracker is in a usable state. However, there are a few other things that I could use help with at this point.
At the moment Bintracker uses Material Design icons, but there’s a few tracker specific things that are missing from that set. For example, of course Material includes buttons for Copy/Cut/Paste and so on. However, in a tracker, there are for example two different ways to Cut. You can either cut a section and leave the following section in place, or you can cut a section and shift up whatever comes after it. Since we want a toolbar button for each of those, this of course requires two different button icons. Paste is even worse. Bintracker supports 4 different paste operations:
Other icon sets I’ve looked at (Feather, FontAwesome, IonIcons, Linea, OpenIconic) won’t be of any help either, if anything they’re even less complete. So there are basically two options: Either derive the missing icons from existing Material Design ones, or design a completely new, custom set of icons. In any case, I’m not the most talented graphician, so I’d really appreciate if someone could help out with this!
Any developers out there who would be willing to help with building Bintracker on Mac? I’m not a Mac owner, so I have very little experience with OS X myself. Building Bintracker itself shouldn’t be too terribly hard. However I’d want to provide a convenient, easy-to-install package to users that will download and set up the necessary 3rd party software (MAME and Tcl/Tk) as well. As far as I know, Tcl/Tk is installed by default up to Mojave, but Apple stopped shipping it in Catalina. Also, ideally I’d want a CI pipeline for automatically building and deploying new releases.
You probably guessed it: I’m not a big fan of social media, and in all honesty I’d much rather spend my time working on Bintracker itself, than promoting it. That said, for Bintracker to become a success, it will be helpful to spread the word as much as possible, especially in regards to crowdfunding (more on that below). So, any volunteers for running a Twitter account and/or spreading the news on Facebook etc.? Please get in touch.
I currently have the privilege to work on Bintracker more or less full-time. However, the good times are coming to an end soon. That means I will either need to take up a regular day job (in which case Bintracker development would obviously slow down significantly), or I need to find another way to generate some income. My initial idea was to run a Kickstarter, but I’m anything but sure that this is a good idea.
The problem with running a Kickstarter is that it can of course fail, in which case I will have put in a lot of work for nothing. And considering that I’d need to raise a quite substantial amount (mind you, a sizeable chunk will go to taxes and Kickstarter fees), plus my sub-par promotion skills, the chance of failure is quite high. I know that I can run a flex-target campaign on Indiegogo, but that would incur even higher fees. Plus, Indiegogo has a pretty bad reputation in some parts of the retro scene. Alternatives? GoFundMe doesn’t accept Paypal, which would put off quite a few people, I’m afraid.
A possible alternative would be to set up a Patreon account. This would carry a much lower risk. However, clearly not everyone who would generally be willing to throw some money my way, would necessarily be willing to do so every month. So if I put my bets on Patreon, I will definitely lose out on some potential income.
So why not try both? Well, then people who opt for Patreon won’t put money into the Kickstarter, increasing the chance of failure even more. And vice versa, people who put money into the Kickstarter won’t feel inclined to put money into the Patreon. So this is a lose-lose situation.
Not sure what’s the best course of action regarding all this. If you have any ideas, please let me hear them. Also, if you would be willing to help with running a crowdfunding campaign and/or setting up other donation options, please get in touch.
Aside from crowdfunding/donations, I’m also open to other funding options. Anybody out there who knows someone at a company that might be interested in becoming a sponsor for Bintracker? With Bintracker being free and open source, there isn’t much I can offer in return, but I would be more than happy to put some sponsor’s logos up on the Bintracker website. I’m also open to the idea of a company hiring me to work on Bintracker. Bintracker could be branded “/YourCompanyNameHere/ Bintracker” in this case. Hiring could potentially be done through a foundation, so any money spent would be tax-deducible, or if the company is doing audio related products, they could hire me directly, allowing me to spend a part of my working hours on Bintracker.
Last but not least, I would also like to get your feedback on the newsletter itself. Is the length OK? Would you prefer shorter, more frequent updates? Is there anything missing? Also, if you have any questions or suggestions for Bintracker, feel free to send them in and I’ll discuss them in the next newsletter.
There are a couple of specific topics that I would like to get your opinion on.
OK, in order to explain this, I’ll need to ramble a bit. Bear with me, or just skip this section if you don’t care.
In my opinion, sequences (aka order lists) are a weak point of trackers in general. The main gripe I have with them is that they interrupt your flow of thought when you’re composing. Also, the fact that there’s no common standard for them in existing trackers seems to underline the idea that there’s something wrong with them. There are two common approaches: Matrix and Unified sequences. By matrix sequences, I mean those where each channel gets its own track in the sequence (like in Famitracker), whereas XM trackers are an example of unified sequences - there is only one track in the sequence, which sets all the channels.
Personally, I’m a big fan of 1tracker’s approach, which ditches sequences completely, and instead uses “sections”. To start a new “pattern” (or rather section) in 1tracker, you just hit the space bar in the “endless” pattern roll. Hit space again to merge back with the following section. This does mean you’ll do a bit more copy&pasting than usual, but overall I think it’s a net win. Copy&Paste requires maybe one or two more key strokes than inserting a new row into the sequence, but this is more than compensated by the fact that I can just compose my melodies in one go, without interrupting whatever I’m doing to edit the sequence. However, that’s just me of course - your mileage may vary.
Now, matrix sequences make sense when the tracker runs natively on the target hardware, because they allow the composer to optimize for size. However, on a PC, optimizing is a job that the computer can do, so there isn’t any good reason to burden the user with that. Either way, here’s the catch: MDAL (Bintracker’s compiler) uses matrix sequences by default, unless specified otherwise by the sound driver configuration. However, those matrix sequences are virtual, in the sense that the compiler rebuilds all sequences during compilation. There are two reasons for this. For one, it makes optimization easier. Secondly, it’s necessary in order to support sound drivers that require a fixed pattern length (like Huby on ZX Spectrum, for example), without bothering the user with such constraints. More importantly though, this opens up some interesting possibilities for Bintracker itself. Essentially, Bintracker could offer the user a choice multiple sequence styles. Want to fiddle with the low-level matrix? Be my guest. Want a simple unified sequence? No problem, MDAL will just optimize away any redundancies that this introduces. Want automatic sequences like in 1tracker? That’s possible, too.
Ok, so I have some questions regarding this. What is your preferred sequence style? Do you actually enjoy working with matrix sequences, or do you find them annoying? What would be your “dream” sequence style? Also, someone at Deadline (sorry, can’t remember your name) suggested to have “blocky” matrix sequences as an option, like in Jeskola Buzz and most “modern” DAWs, where each pattern is represented as a block of a specific size that correlates with its length. I think that’s a great idea. However, if I implement that, I will drop support for classical matrix sequences, as supporting both would mean I’d have to write a lot of extra code. So, let me know what you think - Blocky sequences: yay or nay?
You probably know by now that Bintracker will be heavy on the plugins (and I very much hope I don’t have to write them all myself :D). Anyway, there are two ideas here: first, plugins must be composable, that means plugins can depend on other plugins. Say we have a low-level plugin for version control, let’s call it “git”. Then another plugin, let’s call it “project manager”, could use “git” under the hood to its thing. And yet another plugin, let’s call it “online collaborator” (yeah, being really creative with names here) could in turn extend “project manager”’s capabilities. I’ve got a pretty solid plan for implementing this already, so that part is more or less covered.
What I’m far less sure about is the second idea. Since there are (hopefully) going to be quite a few plugins, Bintracker will need a plugin manager at some point. The plugin manager should resolve dependencies (eg. when the user installs a plugin, the plugin manager should also install the dependencies of that plugin), and it should also do updates and such. Generally I’m thinking to set up a central plugin repository, and the plugin manager should talk to that repository somehow. However, manual plugin installation should also be possible. I’d actually prefer the plugin manager itself to be a plugin (so it could be replaced with alternative plugin managers), though I’m totally not sure whether that’s a good idea. Also, really not sure how to implement the whole thing. First of all, how to make it secure? Hashes? Signed keys? SSL? I don’t have much experience with crypto, so any help or at least some pointers in the right direction would be much appreciated. Also, where should the central repository reside? Bintracker.org would be the most sensible option in my opinion, but that currently runs on a shared web space, so can’t run git on that. As you might know I don’t have a lot of money, so users would have to pitch in for hosting costs, if we were to switch to a real server instead. The other option would be to have the central repository on a service like Github or Gitlab (possibly with a few fallback options). This would simplify things a lot, as many people are already familiar with the workflow of these sites. However, I’m a bit worried about this thing called “vendor lock-in”… Anyway, I’d be glad to hear your thoughts on this.
Well, that pretty much wraps up this newsletter. I was going to include another section with some insight into the technology and implementation details of Bintracker, for the coders among you. However, this newsletter is getting quite long already, so this will have to wait until next time. Once again, if you any questions, suggestions, ideas, or you want to help with stuff - please get in touch.