Day 1

Technically I'm writing this on 2nd of December, but I only learned about this late yesterday. This is the log of my December Adventure journey! In previous years I took part in Advent of Code events to various degrees of success, but as the years passed they started feeling repetitive, competetive, and stressful. I was delighted to find a more laid back approach to December coding, and one that also encourages journaling which I wanted to get into for some time now. Thanks to olivia for sharing this with me!

My goal for December is broadly to work on this server, adding functionality here and there, and specifically at the moment developing my discord bot, Mycelium, which is hosted on this machine. When I'm satisfied with my work there I will maybe take a slight detour to package Factor's vim plugin into nixos and then I will go on to properly set up a blog on this page, with this being the first post. I'm eager to share my work!

Day 2

Mycelium is a personal discord bot with functionality being whatever I care to implement at any point in time. It's implemented in my favourite programming language (Factor), and handles simple commands like rolling dice or fetching a card from a database. For now all of it was based only off of parsing messages looking for a prefix : and an appropriate commands, but I wanted to be able to use the bot in servers where it isn't installed, which means installing it on my user. Unfortunately user-installed discord bots aren't allowed to just scan all messages for a command, so I need to venture into the scary world of discord interactions.

Factor's discord vocabulary has been growing with the needs of its users, and apparently not much need was there for supporting interactions. I will be writing a lot of code myself here, and I hope that in the end I can figure out some useful abstractions to contribute to the vocabulary. To make my work easier I want to reuse as much of my existing code as possible, so I'm adding a single Message Command that will take a message, run it through the same processing as it does when installed on a guild, and print out the results as normal. In fact I just finished implementing this yesterday! If you install Mycelium on your discord user you will be able to "Run" any message as if it was sent on a guild that has Mycelium installed. There are some caveats though...

At the moment the bot will always try to respond to a message command, and always with a text message. This is fine most of the time, but normal commands handle some edge cases: if the response would be empty the bot instead reacts to the command with a 👍, if it would be too large it wraps the response in a file, and if it's not a recognised command at all it just does nothing. Additionally if a message message that triggered a command is deleted the Mycelium response is deleted too as a way to prevent accidentally clogging a channel or avoiding responsibility for inappropriate commands. Current interaction capabilities of Mycelium don't have any counterparts to these features, and it would be nice to reuse the code that does it for normal commands.

Current plan is to use Factor's hooks. They are one of Factor's ways to implement methods, but instead of calling a method of an object on the stack they call a method of an object in a variable. This will let me write alternative behaviors for some words based on whether I am handling a MESSAGE_CREATE or INTERACTION_CREATE event, and hopefully I will be able to write many polymorphic words based on few hooks. The goal for today is to rewrite the current functionality using hooks without adding anything yet, just to check if it makes sense. So to recap, I need to handle these cases:

Day 3

I didn't have as much time as I wanted yesterday, but I still managed to introduce the first hook: authored-by-admin?. In my previous code I use two words: if-admin and when-admin to execute some code conditionally only when I am the one calling the command. They check the appropriate field of the JSON I'm getting from discord for the author of the command and compare it to a configured list of admins (just me). Because the JSON structure of a MESSAGE_CREATE and INTERACTION_CREATE events are totally different, so extracting who is responsible for the event is also very different. That's why I took out the extraction part into a hook that my helper words can use: 

HOOK: authored-by-admin? last-opcode ( -- ? )

M: MESSAGE_CREATE authored-by-admin?
  discord-bot get last-message>> obey-message? ;

M: INTERACTION_CREATE authored-by-admin?
  discord-bot get last-message>> [ "member" of ] keep or
  "user" of "username" of
  discord-bot-config get obey-names>> in? ;


: if-admin
  ( ..A then: ( ..A -- ..B ) else: ( ..A -- ..B ) -- ..B )
  authored-by-admin? -rot if ; inline


: when-admin ( ... quot: ( ... -- ... ) -- ... )
  authored-by-admin? swap when ; inline

That was yesterday, but today I moved all of the message sending code to hooks! Now both the handlers for MESSAGE_CREATE and INTERACTION_CREATE use (almost) the same words to handle the command and respond. The main responder is sized-message* which dispatches into different subresponders based on the size of the response string. Each of these subresponders is a hook that does slightly different things based on where the command came from, but I tried to reuse the same abstractions between them as much as possible. Unfortunately that's not as much as I would have liked: there are a lot of similarities between all the responders but they're so miniscule that abstracting them away would only make everything more messy. I'm sure I haven't arrived at the best design, but I don't imagine the optimal case will be very satisfying. Hopefully as I work on this more I will be able to iterate and improve my code.

Day 4

Today I took a little detour from working on Mycelium. I will be running a discord game of Blood on the Clocktower with a custom script, so I need to gather up descriptions for all characters and format them in Markdown to post on discord, like this: 

* [Name Of Character](https:/url.to.the.wiki/Name_Of_Character): Description of what the character's ability is in the game
All of the links and descriptions are already available on the game wiki, but 25 characters on the script I decided that looking up each character and copying its url and description is too much repetitive work that could be automated. I made my first site scraper.

I've always heard that site scraping is an ugly and inconsistent work. A scraper usually has to rely on the site having specific structure, so if the structure is inconsistent or changes the tool will break. Fortunately I only want to scrape a pretty small piece at the beginning of an article, but even that caused some problems.

Generating the URL from a character name is simple: just replace all spaces with underscores and prepend the wiki domain. Then a simple GET request fetches the site's contents. I was interested in the first part of the "Summary" section, the text encased in quotation marks. Inspecting the HTML on a couple wiki pages showed that the section always starts with <h2><span class="mw-headline" id="Summary">Summary</span></h2>\n<p>" so I wrote up a short regular expression that should do the job: R/ Sumary<\/span><\/h2>\n<p>".*"/. This worked fine to generate descriptions of all Townsfolk on my script, but there were some errors when it came to scraping Outsiders: the regex failed on the wiki page for Ogre. I decided that it must be because the description is long enough that it contains a newline, so I added the s option to my regexp so that . can match newlines as well. This promptly backfired, because in Factor all quantifiers are greedy so the regexp started matching almost entire pages, up to the last occurrence of ". To circumbent that I changed it so that only the start of the description was found using regexp, and then I simply cut it off at the first ", which fixed it for all characters except for the Ogre! Turns out that the Ogre summary doesn't close its short description with a ", it does so with a ”! In the end I rewrote the regexp to handle both quotation marks at the end but not use the s option, and it correctly generated me all descriptions I needed for the script.

The final code is short but definitely not long-term reliable. It depeds on a bunch of assumptions:

That being said I think the fail cases are handled well enough that if one ever happens it should be easy to just copy it by hand as a special case without much problem. This was fun, but I definitely wouldn't want to make anything more complex than this when it comes to web sraping.

Day 5

Today I had a job interview and I do NOT want to think about it more than I have to.

I did read up on buttons in discord messages though. They're what's called "message components" and they generate interactions without needing to register a command. My plan is to make Mycelium send an ephemeral message when ran from a message interaction, and add to it a button that would resend that message so that it's visible to everyone. This is a counterpart to removing responses to commands ran from a guild installation.

Day 6

I figured out how to send messages with buttons! Apparently a message can't just have a button as its component, it needs an action row which itself can contain a button. This, of course, isn't documented on the relevant page and the error response mentions nonexistent component types in the allowed list. Discord isn't the worst system I've worked with documentation-wise, but it's certainly not great.

Along with buttons came some refactoring of how my code generates multipart/form-data messages for file uploads, and more of that is coming in the near future. I finally got rid of hardcoded JSON strings because with the addition of buttons they got too unweildy and incomprehensible, and it let me get rid of one ugly hook. With all that normal commands and context menu commands correctly produce file attachments when the output is large, but unfortunately it doesn't work with the public messages sent after the "Show" button is pressed. I have some idea for how to handle those cases, but all of them will require more refactoring of my message-sending words.

Sidenote: yesterday a draft of the Zen of Factor has been posted on the Re: Factor blog, and I think it's beautiful and conveys the feeling of working on Factor and in the Factor community excellently. One line in particular perfectly describes my experience working on Mycelium:

First make it work, then make it beautiful.

Day 7

Busy weekend ahead! Not much programming will happen probably. I have two separte concerts (I'm a tenor in several choirs) and they will take up a lot of my time. That's fine though, because implementing correct responses to interactions will take some thinking.

Currently the Mycelium database has only one table: INTERACTION, which holds IDs of messages that mycelium interpreted as commands and her responses to them to delete the resposnes when the original message are deleted (and same for editing in the future). I came up with that name before considering supporting user installation of Mycelium, and it's very unfortunate now that I want to record the message interactions. There's a lot of inconvenient name collisions all around, another one being that would like to save "responses" to command messages, but there are already HTTP responses among the loaded namespaces. I'll figure something out eventually, but renaming a table in the database is the most annoying thing so I want to have these without collisions ASAP.

Day 8

I moved the INTERACTION table to a RESPONSE table and renamed its ORM counterpart to command-response. This is still not the perfect name since outputs coming from an interaction are also supposed to be responses, but it will do. In fact the ORM for the interaction counterpart of this table I called interaction-response, and it's being mapped to a brand new INTERACTION table. That's all for today, and it's just to fix a tiny issue: when the "Show" button is pressed on an ephemeral message the public response sent from it is considered a response to the even of pressing that button, so it references the ephemeral message instead of the one that included the command. This looks like the output is from a deleted command for other people. By persisting the interaction token I will be able to link the public message back to the original message including the command.