BINTRACKER
03 Feb 2023
Winter time is Bintracker time! After a busy summer and fall here on our homestead, I finally found some time to work on the project again. And this time around, I’ve got some pretty exiting news to share.
It’s been an extraordinarily challenging and exhausting process (more on that below), but it’s done: We have a working Windows build. Head over to https://github.com/bintracker/bintracker/releases to check it out. A little refresher on what’s currently possible in Bintracker:
Note that the Windows build does not contain the ROM files required by MAME - read this guide for details on how to add those.
Bintracker is definitely still rough around the edges, so be prepared for frequent crashes and other nasty surprises. Save often, and most importantly, please report any bugs you come across on the issue tracker or by sending a mail to the address you’re receiving this newsletter from.
Also, big thanks to AtariTufty, who was brave enough to test the pre-release.
Aside from getting Windows builds off the ground, I’ve mostly done bug fixes. Also, the latest mkdocs and mkdocs-material 9 broke the documentation build, so I spent some time on getting all of that up to date and running again. Still not 100% there yet - if anybody wants to look into customizing the color theme that does not involve hacking the main css, that would be great. The officially prescribed method of putting an extra.css in docs/stylesheets does zilch. In hindsight, I’m not sure if choosing mkdocs was the right decision, as this isn’t the first time an update broke the build. I hope to run everything off scm2wiki one day, but that day is still far off, unfortunately.
I also temporarily removed the PhaserX-MZ700 engine, as it has some stability issues that I haven’t been able to fix yet. Instead, I’ve added a new engine for the Fairchild Channel F. Sleizsa Quartet is the successor to the trusty Sleizsa Trio engine, featuring 4 channels, variable duty cycles, PWM sweep, and PWM sampled drums. I’ve had this laying around since 2019 - happy to be finally able to use it!
This is the part where I rant about the horrors of the whole Windows build experience, so feel free to skip ahead if you aren’t interested in the nitty gritty details.
My original plan was to use MinGW to cross-compile from Linux. After a lot of hunting down obscure dependencies, kicking and massaging the makefile, and switching to another PNG library because I couldn’t get MinGW to build imlib2, I actually managed to get the project to build. There was only one small problem - on Wine, the program would crash after startup, and on real Windows, it wouldn’t start at all. Some Chicken Scheme experts suggested to increase the executable heap size, which up to certain point made the program crash… slower. Beyond that point, Bintracker would run on Wine - but crash with a STATUS_BAD_STACK error on actual Windows. Who’d have thought that nowadays Wine is actually better at running dodgy Windows software that Windows itself?
Anyway, I abandoned cross-compilation and set up a Win10 virtual machine with VirtualBox instead. Now, using MinGW on Linux is not exactly pleasant, but compared to MinGW on Windows, it’s heaven. On Windows, nothing works out of the box. Installation doesn’t set up $PATH properly, and it’s not like simply adding MinGW’s bin directory will do. Also, why does MinGW on Windows have like 10 different shells? How am I supposed to know which one I need? Well, let’s have a look at MinGW’s documentation. Oh. Ok. I mean, I know Bintracker’s documentation isn’t exactly stellar at this point, but this… well, never mind. Long story short, eventually I managed to build Chicken Scheme. And sure enough - the interpreter works, but the compiler doesn’t, because something something not found error lol.
Tired of trying to coerce plain MinGW into producing a workable Chicken compiler, my next step was to try and build Chicken on MSYS (using MinGW under the hood, of course). My memory of this is hazy - in any case, it didn’t work. Then, while searching for solutions on the web, I stumbled across Chicken Scheme the Chocolatey package. I had nothing to lose at this point, so I gave it a try. And it actually worked. Sure, it’s an old Chicken 5.2, but hey, I’ll take it. Massive props to package maintainer zilti. From here on, it was almost smooth sailing for a while. SQLite installed like a charm in MSYS. Chicken’s SRFI-18 multi-threading library wouldn’t budge at first, but I managed to get it to work by tweaking its build script at bit. And at last, I was finally able to build Bintracker again. Even better, that build would actually start up without complaints.
Of course, I wasn’t done yet - pop-up dialogues weren’t working, and once I fixed that, it turned out that the way Bintracker was communicating with MAME under Linux wouldn’t be possible under Windows. Around that time, the VM started to act up as well, complaining about hard drive failures (yeah right) and eventually refusing to boot. Farming season was starting up by then as well, so I put the project on hold for the time being.
When I returned to it in November, I could no longer get a working Windows VM in VirtualBox. Tried VMware, alas without much success, as I’m on a non-systemd Linux distro, and VMware is pretty much married to systemd nowadays. I looked into setting up a VM with KVM/QEMU, but MS doesn’t provide any evaluation images for this setup, and at my income levels I’m not inclined to shell out money for a full Windows license. So I was about to give up when I stumbled across Vagrant, which can run on top of KVM/QEMU and has a Win10 VM image in its catalog. And holy cow, does it work smoothly. Traumatized by VirtualBox, I didn’t dare to imagine that virtualization could ever be pleasant. No random crashes, no wonky proprietary “Guest Additions” (not that it matters much on a proprietary operating system), just rdesktop and rsync and off you go. Brilliant.
So, off I go installing the toolchain via Chocolatey again, installing the libraries, when suddenly -whoops- the check-errors library won’t build and actually completely borks the chicken-install utility in the process. How come that didn’t happen earlier? Turns out check-errors got an update in the meantime, which for some reason doesn’t work under Windows. It’s a low-level dependency that I definitely can’t do without in Bintracker, so I needed to fix this somehow. Took me several days to find the problem - the library actually builds and installs fine, but fails the final step of creating an .egg-info file, which consequently sends chicken-install to chicken-hell. After manually creating that file in the appropriate location, everything works as intended again.
Surely, our odyssey must be over soon. Just need to figure out a way to talk to MAME on Windows… So how about using TCP? I’ve been meaning to add TCP support anyway, as this would open up the path to supporting some emulators other than MAME. Chicken Scheme comes with a TCP module, so implementing TCP on Bintracker’s side was easy enough. Unfortunately, doing it on MAME’s side proved to a little more… challenging (for the inept developer that I am, in any case). MAME scripting happens in Lua, and in order to make Lua speak TCP, you need the luasocket library. And in order to build that, you first need Lua on your system. Hoping I could spare myself the trouble, I hunted around for binary builds of the library, which are surprisingly hard to come by. Eventually I found some, but they were built with Lua 5.1, whereas MAME runs Lua 5.3. So, I took a deep breath, set up a Lua toolchain, and, after a bit of makefile massaging, was able to build the lib. Just a slight problem - when importing the library into MAME’s Lua interpreter, it goes looking for either a core.dll or a socket.dll. Great, except that’s not what the luasocket build outputs. Can’t remember everything that transpired afterwards, but rest assured I tried a great many things, all of which ended in some variation of the by now all-too-familiar “module socket.core not found” error. Eventually I went hunting for binaries again and this time managed to find an actual Lua 5.3 build. Did it work? Well, take a guess. Turns out, you don’t just need to build against the correct Lua main version, you need to build against the exact point release. Well, maybe, that is. I never managed to get luasocket working in MAME, but from reading various forum posts it appears that some people have (and a great many more failed). Either way, one day I happened to browse around in the source code of MAME’s Lua interpreter, when I stumbled across this gem:
* file:open(name) - open first file matching name in searchpath. supports read
* and write sockets as "socket.127.0.0.1:1234"
You’ve got to be kidding me! Indeed, as it turns out we don’t need luasocket to make MAME speak TCP after all. Now the ordeal is almost over. I made one more mistake that cost me another couple of days, likely due to my inexperience with networking shenanigans. My plan was to implement things so that MAME would be the client, and Bintracker would be the command&control server. My reasoning was that Bintracker has no means of knowing when MAME is done booting, so MAME should contact Bintracker once ready, and request further instructions. Turns out that this probably doesn’t work with MAME’s emu_file implementation - in any case, I couldn’t get it to talk to the listener on Bintracker’s side, even though I could connect to it with another client just fine. So instead I made Bintracker the client, and just let it mindlessly pound the listener on MAME’s side until that one finally answers. And with that, I finally had a working Windows build. Yay, that was fun. NOT. I guess an experienced developer could have figured this out in a day or two. Too bad I’m not an experienced developer, but just some guy trying to make music on some obscure 40 year old junk.
At this point Bintracker is still firmly in alpha stage. I would like to get the project to beta stage as soon as possible. Therefore, the plan for now is to hold off on implementing any major new features, and instead focus on improving the overall stability of Bintracker (and of course managing the onslaught of incoming bug reports from you all). That said, there are a number of things I would like to tackle eventually. In no particular order:
I also still need to set up continuous integration. Any help with that would be greatly appreciated. Not sure yet which route to take - I very much don’t want to rely on Github for this, but on the other hand Github Actions seems to be the easiest way nowadays. Anyway, we’ll see how far I get before the farming season starts up again. I will definitely spend more time on Bintracker in the coming weeks, but I also have a few other projects that have been on hold for much too long.