Experiments with Gradual Typing in Ruby – Part 1

I recently started toying with Gradual typing in Ruby.

Of course, Ruby is already typed: messages are dispatched to objects depending on their type. That’s dynamic typing, enforced at runtime.

But for larger programs, it can be useful to have some static type-checking, that can be enforced by a type-checker without running the whole program.

Enter type annotations. By adding some explicit informations about the expected types in our program, a type-checker will be able to catch some errors using a static analyzer.

What does it look like?

Gradual typing is not standardized in Ruby yet (although some efforts are ongoing). So there are different tools available. Currently, the best way to add gradual typing to Ruby programs seems to be Sorbet.

Here’s a simple Sorbet example, using an existing Ruby function:

def to_hex(i)
  "%x" % i
end

We can add type informations to this function using the sig function decorator:

extend T::Sig

sig {params(i: Integer).returns(String)}
def to_hex(i)
  "%x" % i
end

Granted, the syntax is a bit weird. But it has the merit of being valid Ruby code, which allows it to be accepted by the standard Ruby parser without modifications. And like many syntaxes that seem unusual at first, our eyes quickly get used to it (hi, Objective-C square brackets).

Letting “Gradual” shine

Now, the neat thing with gradual typing is that you don’t have to provide type informations everywhere. This is useful in many ways.

First, you may start adding type checks to an existing codebase. In that case, declaring all types from the start can be a daunting task. Fortunately, in the absence of types, the type-checker will consider that we know what you’re doing. Which means we can start adding a few types here and there, and already have useful type-checks – but the parts of the program which are still type-annotations-free will not results in warnings or errors.

Second, a lot of Ruby elegance and fun comes from its dynamic nature. Sometimes the most elegant way to express some code is to use dynamic method resolution, or other dynamic-oriented constructs which cannot be type-checked. And this is okay! In that case, gradual typing means you have an escape-hatch: as types are not mandatory, they just won’t be type-checked. We can get the benefits of type-checking for 95% of the code, and still use neat dynamic features in the remaining 5%.

And last, at times it can be useful to just experiment and prototype some code quickly, to see how the structure would look. In these cases, you won’t have to fight your way through types: you can just omit type annotations, and quickly let the code flow without thinking too much about production-ready reliability.

Experimenting with a standalone Ruby script

I’ve never used Sorbet before, so I wanted to start small, and get used to the type system.

Fortunately, the Sorbet website provides an online playground: just type in some Ruby code, add some type annotations, and the type checker will start telling you what’s right and wrong with the types you provided. Neat.

I used a standalone Ruby script I wrote some times ago. This script reads assembly source code and debug symbols from the local filesystem, and infers from this the probable location of more debug symbols. Here the initial version of the script, and the final version after finishing adding types.

When I copy-pasted the script, without adding anything yet, Sorbet immediately told me about two errors: <function> is not available on NilClass (https://srb.help/7003).

Wow: it detected that in two different places, my code was sending a message to a potentially nil object. And gave me an URL to learn more about the issue.

How to fix this? I followed the URL, and a well-written document explained me that I could either:

So the two reported errors were easy to fix. I was quite impressed that Sorbet found two relevant mistakes without even starting to add types. And even more impressed that it not only does nil-checks, but also type propagation (that is, when some code checks if a value is nil, Sorbet considers that after this point the variable can not longer be nil).

Adding types

After this, I started adding type annotations to a single method (a constructor). That was easy enough: just a matter of adding the correct sig {…} incantation.

But right after that, Sorbet told me about a new error: my signature stated that the method argument was a String, but elsewhere I was calling the constructor with a T.Nilable(String) – that is, an object that may be nil. Interesting. Like before, to fix it, I had to add the proper nil check before calling the constructor.

I then gradually added type annotations to more methods, and found it almost fun. I had the feeling that I was strengthening my program, and uncovering the hidden assumptions that had been there before.

All of this went rather smoothly (except having to convert Ruby Structs, unsupported by Sorbet, into T::Structs). The weird syntax quickly became bearable, and eventually even read like being a part of the documentation.

Refactoring

In the end, this even led me to write better code. For instance, consider this function :

class Address
  def self.from_string(str)
    bank, offset = str.split(':')
    self.new(bank.to_i(16), offset.to_i(16))
  end
end

It extracts the two components of a semicolon-separated string – like 03:4A2F.

When adding type annotations, Sorbet initially told me “Hey, offset.to_i(16) is not valid on NilClass”. Because of course, it detected that if the input string is badly formatted, offset may be nil.

So I quickly wrapped the value in T.must(…), to silent the warning away. After all, there’s not so much we can do to prevent badly-formatted input; crashing at runtime seems a sensible option.

class Address
  sig {params(str: String).returns(Address)}
  def self.from_string(str)
    bank, offset = str.split(':')
    self.new(T.must(bank).to_i(16), T.must(offset).to_i(16))
  end
end

But wait, there’s better than crashing at runtime: and that’s “crashing at runtime with a meaningful error message”. What if we rely on nil-propagation to write instead:

class Address
  sig {params(str: String).returns(Address)}
  def self.from_string(str)
    bank, offset = str.split(':')
    raise "Invalid address format" if bank.nil? || offset.nil?
    self.new(bank.to_i(16), offset.to_i(16))
  end
end

Nice: a bad input now gives us a readable error message. And we can even remove the T.must checks, because, thanks to nil-propagation, Sorbet is now sure that offset.to_i(16) is not called on nil.

So far

After toying with the Sorbet’ playground, here are my first impressions:

The good

The bad

The ugly

What’s next

For now I haven’t tried to type-check a program locally, nor to type-check code that relies on external gems.

So my next step is probably to add some minimal type-checking to a small Rails app, and see how Sorbet’s tooling deals with the many dynamic constructs of the framework.

La Zone du dehors et le management par objectifs

Il y a quelques mois, j’ai fini La zone du dehors d’Alain Damasio. J’en suis ressorti avec des impressions mitigées, qui ont fini par se décanter. Pour une bonne partie, j’ai eu l’impression que la narration avait du mal à savoir où elle allait, que les séquences se juxtaposent dans une certaine confusion thématique – et que même si j’ai du mal avec les arcs narratifs lisses et stéréotypés, un peu de cohérence ne fait pas de mal non plus.

Mais y’a quelques points qui m’ont bien intéressés. En vrac :

Le vitalisme

Une des choses que La zone du dehors a clarifié pour moi, c’est que Damasio me semble attaché moins à l’anarchisme qu’à un genre de vitalisme. “Oui, tout foisonne, tout bouillonne, ça pose plein de problèmes mais c’est génial”.

J’ai l’impression que cette appréciation du vitalisme se trouve aussi dans Les furtifs, et dans une moindre mesure dans La horde du Contrevent – mais aussi beaucoup dans ce que Damasio a pu écrire sur la zad de Notre-Dame des Landes il y a quelques années.

Et ça m’aide à comprendre pourquoi tout ne me parle pas dans les récits de Damasio : le vitalisme c’est sa came à lui, et pas trop la mienne. Et que y’a aussi d’autres choses que ça dans l’anarchisme (genre des courants qui sont plus “l’ordre moins le pouvoir”). Et que c’est bien aussi.

Le patriarcat

Ouch. La zone du dehors, en terme de féminisme, c’est tendu. Et j’ai l’impression que c’est un point aveugle des écrits de Damasio depuis longtemps.

Je renvoie ici à la série d’articles (encore en cours) de Mélissa et Lunar, qui ont fait une analyse des Furtifs sous ce prisme, avec plein de dataviz : https://dérivation.fr/furtifs/.

La société de l’évaluation

Les passages sur la société de l’évaluation m’ont bien parlé. En bonne partie parce que pour moi ça donnait du sens à ce qui se passe dans le néo-management qui se retrouve fréquemment dans la startup-nation.

Vous savez, c’est cette forme de management à la cool où l’évaluation est omni-présente. Toutes les deux semaines, un one-to-one avec un manager. Toutes les six semaines, faire une liste d’objectifs hyper-précis à atteindre – et définissez des cibles chiffrées. Tous les six mois, une “review à 360°” où vous êtes évaluées par des managers et des pairs – dans l’espoir d’obtenir une augmentation qui ne viendra jamais. Et en permanence, devoir participer aux nombreuses évaluations de vos collègues. Mais c’est cool, parce que c’est vous-même qui définissez vos objectifs, tout ça.

Évidemment, tout ça ressemble fortement au Clastre, le grand système de classement social général de La zone du dehors. Et ce qu’en dit Damasio dans la grande tirade deleuzo-foucaldienne, c’est que l’objectif de tout ça est de re-former la personnalité des individus.

Il s’agit bien sûr d’une part les individualiser (chacun essaie de suivre ses objectifs le nez dans son guidon, ça atomise), mais même au sein de l’individu, de découper chaque individu en compétences, en traits de caractères désirables, de diviser l’individu lui-même. Et ensuite l’évaluation des traits permet de dissoudre en partie l’individu, et à la place d’encourager les traits désirables par l’entreprise / l’État / etc.

Au même moment, dans ma vie, j’avais eu vent de discussion dans une startup de la fintech française où des managers commentaient des traits de personnalité. Par exemple « Finalement, toi tu sais défendre tes idées, mais tu respectes aussi l’autorité quand il faut », des choses de ce genre.

Mais en fait c’est des discours complètement performatifs : ça sert pas à remarquer ce qui est bien, mais à dégager un trait, et à dire aux individus « Ça serait bien que tu cultives ça, c’est valorisé par le management, ça t’aidera à avoir une augmentation aux prochains 6 mois. »

Bref, cette idée de “Le micro-management (même auto-organisé) sert à non seulement individualiser, mais séparer les individus eux mêmes en plusieurs traits, et à les modeler” ça m’a semblé éclairant.

The hidden structure of Link's Awakening Overworld map

This article is part of an ongoing “Disassembling Link’s Awakening” series, where I attempt to gain some understanding on how special effects were implemented in this game.

The Game Boy, as most gaming platforms at the time, uses tiles for graphics rendering. Instead of drawing individual pixels on the screens, games must first provide tiles (small graphics fragments, usually 8x8 pixels wide), and then combine together this limited number of tiles to draw pictures on the screen.

This technical limitation influences not only the games appearance, but also the gameplay. This articles goes in-depth into a specific example: how the design of Zelda: Link’s Awakening overworld map is influenced by the underlying tile-based rendering.

Challenges of tiles-based rendering

To display an image on screen, as all tile-based games, the game engine must first copy the required tiles to the Game Boy VRAM.

This brings two challenges:

The tileset of Link's Awakening Overworld
A part of the tileset used on the Overworld.
This is way to big to fit all in memory at once, so only parts of it can be loaded at a time.

So in order to display graphics efficiently, great care must be given to the ressource-management of tiles.

A modern game engine would probably include a ressource-management system, which would ensure that, for each frame, the tiles required to display the objects on the screen are properly loaded. But the code for such a system would be quite complex: as uploading tiles takes a lot of time, it would have to predict which objects are going to appear on screen. The system would also have to ensure that not too many different objects are going to be visible on screen at once–otherwise there’s no space left in VRAM to upload the required tiles.

But on older hardware, such as the Game Boy, VRAM is so limited that every single tile must be put to use. There’s no margin left for predictive loading or fancy resource-management systems.

Instead, older games generally use tilesets. Tilesets are some fixed sets of tiles that are grouped together, and known to be predictable available at a given time. The game designers typically give each scene its own tileset, and then use a tile editor to draw the rendered frame. And the tilesets can be switched when the game transitions from one scene to another.

As of Link’s Awakening, the code responsible for this ressource management has recently been documented. And this is exactly how the game manages tiles. The world map is divided into sections of 2x2 rooms. Each section has an associated tileset, which allows the map feature some variety between the different sections (because they can use different tilesets).

OverworldTilesetsTable::
    db   $1C, $1C, $3E, $3C, $3E, $3E, $3E, $30
    db   $0F, $36, $36, $1A, $0F, $34, $0F, $3E
    db   $20, $20, $0F, $38, $28, $28, $32, $32
    db   $20, $20, $38, $38, $28, $28, $32, $32
    db   $0F, $26, $0F, $24, $0F, $1E, $2A, $0F
    db   $26, $26, $2E, $2E, $0F, $2A, $2A, $2A
    db   $0F, $24, $2E, $2E, $3A, $0F, $26, $2C
    db   $22, $22, $22, $0F, $3A, $3A, $0F, $2C

An array is not very telling. Let’s use a picture instead! And we’re lucky, because a few years ago, Xkeeper generated a map of the Overworld with the tileset IDs overlaid on each section.

Overworld map of Link's Awakening, with the tileset ID overlayed on each map section Each 2x2 section of the Overworld map declares its own tileset ID. (Credits: Xkeeper)

Quite simple – in theory.

Technical constraints

Practically, it’s not so easy. As we know, in Link’s Awakening, when player moves from one room to another, the game animates the changes smoothly. Which means that during the transition, both the previous room and the next one are visible.

So during a room transition, both the old and new tileset need to be available in VRAM. Otherwise, glitches will ensue: if the tileset of the new room overlaps the tiles of the previous room, the previous room will shortly be rendered with incorrect tiles. Every time the game designers want to introduce new tiles, they have to think about the transitions from all the adjacent rooms. Moreover, this is hard to debug: the glitches could manifest only when visiting the rooms in a specific order.

Link’s Awakening engine has a solution for this: put structural constraints on the map design to avoid the issue almost entirely. Instead of testing every combination of rooms and tilesets, the game instead:

  1. Defines a special “Keep current” tileset code,
  2. Ensures that the player always goes through a “Keep current” tileset before loading a new one.

The “Keep current” tileset code is a special tileset ID, that instructs the game engine not to load any new tileset data for the current section of 2x2 rooms. You can spot it on the tilsets map above: it has a 0x0F ID.

How does it solves our tileset issues? Well, a “Keep current” tileset is kind of a buffer zone between two different tilesets. A section with this tileset must be displayable using any of the tilesets of the section leading to it.

For instance, given the following map layout:

 ——————————      ——————————————     ———————————
| Section 1 |   |  Section 2   |   | Section 3 |
|           | ↔ |              | ↔ |           |
| Tileset A |   |"Keep current"|   | Tileset B |
 ———————————     ——————————————     ———————————

In this layout, depending on the player direction, the Section 2 tileset may be displayed either with Tileset A, or with Tileset B.

Which means that by design, a “Keep current” section can only use the tiles shared by the adjacent tilesets, in that case the Tilesets A and B. If this rule is not honored, the graphics of the “Keep current” section will be corrupted.

So by introducing some design constraints, the game avoids to ensure that every tileset has to be compatible with any tileset that could be adjacent to it. Now the game designers just have to ensure that some sections only use a restricted number of tiles shared between the adjacent tilesets. These “Keep current” sections will feature less unique details, but will ensure that transitions between different tilesets are always glitch-free.

Revealing the hidden structure of the Overworld map

The only thing that remains is to ensure that the player can never navigate directly from a tileset to another – but instead always goes through a “Keep current” area first.

Lo and behold, this reveals the hidden structure of Link’s Awakening Overworld map.

Here’s the same Overworld map than above, with the tilesets overlaid – but this time, the “Keep current” tilesets are highlighted in green.

Overworld map of Link's Awakening, with the "No change" tileset overlaid in green Overworld map of Link’s Awakening. In green: the “Keep current tilesets. In red: walls and natural obstacles on the map.

As you can see, the game designers had to put restrictions to ensure that the player can never directly transition from a tileset to another – but instead goes through a “Keep current” tileset first.

How? By putting walls and obstacles on the map that separate the tilesets. These are the one highlighted in red on this map; the player can never go through them. The obstacles constraint the player’s path, and ensure the tilesets continuity.1

Impact on game design

Like all technical restrictions, limitations on tilesets are also a source of creativity.

Because of the “Keep current” buffer tilesets, it’s easier not to connect every section of the map to every other: obstacles must be built. But this constraint has upsides: it gives the map a labyrinthine structure. And that’s helpful for a good game design: it tends to divide the map into distinctly themed sections. Folds also make the world feel larger, like does a curated garden with carefully placed occluders.

The need to interleave buffer tilesets on the map also gives natural pacing to the game. Buffer tilesets can’t have the same visual complexity than other sections of the map (because they can only use a limited number of tiles). It makes simpler areas alternate with higher-complexity ones. As a result, the player will usually travel through a strongly-themed section, then a more generic one (as they move through a buffer tileset), then again reach another themed section. This sense of rhythm is a key element of a good game design.

Want to read more? Have a look at the other articles of this series, discover more of the code, or join the discussion on Discord.


Notes

  1. Of course there are several exceptions to this:

    • The Photographer Shop section (using tileset 0x1A), is actually a “Keep current” tileset. But when the Photographer was added in the DX version, the tileset was special-cased to load the shop tiles when entering this room. On other rooms, it behaves like a true “Keep current” tileset.
    • On the Windfish Egg section, tileset 0x3C can communicate directly with the tileset 0x3E on the East. For this, the room on the right of the Windfish Egg is special-cased to swap the tilesets smoothly.
    • Around Kanalet Castle, some other sections do change tileset without going through a “Keep current” tileset. In that case, we’re back to manual tiles management: the connecting rooms are carefully engineered to use the overlapping parts of the two tilesets.

    So the constraint of buffer sections allows to greatly simplify tilesets management – but is still flexible enough to allow exceptions, or even revert to manual tiles management wherever needed. 

Link’s Awakening disassembly progress report – part 11

This article is part of an ongoing “Disassembling Link’s Awakening” series, where I attempt to gain some understanding on how special effects were implemented in this game.

✨ New contributors

First let’s congrats the following new contributors, who made their first commit to the project during the past months:

🔀 Source-code shiftability

Efforts to make the disassembled source code shiftable have been ongoing for half a year now. And this time, there’s a great news:

Zelda: Link’s Awakening source code is now shiftable! 🥳

What does that mean? Let’s borrow an analogy from Revo, of the sm64decomp project:

You have a piece of graph paper on a table, except there’s a problem: nails are nailed through certain coordinates to the table. Try to move the paper and it tears it up.

Shiftability means pulling each nail and writing down the coordinate it was pointing to.

Binary modders are just really good at working with the nailed paper, regardless of the nails, by cutting and pasting available pieces and drawing on the empty spaces and stitching stuff (and in some cases just tapes on an extra piece of paper which is used as a dumping ground).

For more details about shiftability, see the previous disassembly progress report.

Getting the source code shiftable means that we reached the point where all nails are removed. Which means it is now much easier to add new code, or change what the current code is doing–without breaking the game. For instance, it could make it easier to create a level editor, a full-conversion mod, a gender-swapped version of the game, or a randomizer.

Building tools to help pointers resolution

This was a months-long effort, that required scanning 150,000 lines of code for hardcoded pointers. To help this effort, some of it was automated.

ℹ️ This section dives deeper into the technical details of pointers resolution. If you prefer a higher-level view, jump directly to the next section.

Most pointers can’t be reliably identified using purely automated ways: a series of bytes like db $01, $4E, $87, $4F could be the definition of two pointers ($4E01 and $4F87), but might as well be a display list, or any data block.

However, load instructions (such as ld hl, $65E2) almost always refer to a data pointer in the current bank. This is not 100% foolproof: sometimes the loading instruction may be data that was wrongly interpreted as code, or it may be loading a pointer to be used later in another bank. But mostly, this is a good guess.

Using these guesses, a script scanning the source code for these loading instructions was written. Using the current state of the source code, and the symbols already identified, the script can output a list of missing data pointers for the current bank – including the length of the data block.

The way the script work is:

  1. Read all the symbols already generated;
  2. Read the given source file (e.g. bank_1F.asm);
  3. Find all the ASM loading instructions in the source file that refer to a raw pointer (e.g. ld hl, $65E2);
  4. Convert the raw pointers to data symbols in the current bank (e.g. $652EData_01F_652E);
  5. Guess the size of the data blocks, by assuming the blocks runs up to the next symbol defined in the bank;
  6. Emit a new set of debug symbols, augmented with the newly extracted pointers;
  7. Run the disassembler again, feeding it the augmented symbols.

Once the disassembler is run again, it produces an updated version of the source file–but with the data blocks properly labeled, and with those labels correctly referenced the loading instructions.

This new source file must then be manually merged with the original source file, by cherry-picking the data-label changes, while keeping the annotated comments from the original files.

In the end, this script helped to resolve around 50% of the raw data pointers. The rest of it (jump tables, pointers tables, etc.) had to be labeled manually.

Ensuring shiftability

If some hardcoded pointers remain in a code bank, adding or removing code from those banks won’t shift those pointers. This could result in subtle bugs in the compiled game.

To ensure that a given code bank is shiftable, a simple check is to insert some nop instructions at the beginning of the bank (thus shifting the entire code by some amount), and play the game to see if anything breaks.

This method is simple–but running all code paths to ensure that everything works smoothly is difficult and time-consuming. Moreover, once a bug is found, it can be tedious to identify the precise location of the faulty pointer.

@marijnvdwerf found a better way: he used other versions of the game. Three main different versions of Link’s Awakening DX were released: v1.0, v1.1 and v1.2 (and this doesn’t count smaller changes localized versions in English, French and German). While attempting to add support for other versions, Marijn found many small non-matching data blocks. Indeed, these data blocks contained raw pointers, that weren’t properly shifted when compiling another version of the game.

Luckily, unmatched data blocks are easy to pinpoint–for instance using an hex diffing tool. The pointers still had to be fixed manually, but at least they were precisely identified.

Caveats

Although the code is shiftable, some of the graphics data may not be moved around freely: it requires splitting large graphics sheets into several smaller pieces, which is not entirely done yet.

But shiftability is definitely a huge milestone, and should make the life of moders easier.

🎵 Music disassembling

Around March, @Drenn1 started to have a look at the format of music tracks. Rather than documenting the code, he was attempting to understand the meaning of the music data.1

This led him to write an impressive Python script, that can read the music track binary data, and dump them in a human-readable form.

Long-story short, a music track is defined by:

  1. A default transposition factor (usually 0);
  2. A default speed;
  3. Up to 4 channels tracks.

The 4 channels tracks are controlling the 4 hardware audio channels of the Game Boy: two square wave (that produces MIDI-like music), one programmable waveform (for playing custom sounds), and one programmable noise generator (for playing noise-based SFX).

And each channel track is actually a program. Channels tracks are a sequence of opcodes, that can either:

For instance, let’s have a look at this channel track:

ChannelDefinition_1b_50ab::
    set_envelope_duty $a0, $84, 2, 0
    notelen 4
    note A#3
    notelen 2
    rest
    note A#3
    note A#3
    notelen 1
    note A#3
    note A#3

    begin_loop $02
        notelen 6
        note A#3
        notelen 1
        note G#3
        notelen 3
        note A#3
        notelen 2
        rest
        note A#3
        note A#3
        notelen 1
        note A#3
        note A#3
    next_loop

    ; snip…

This channel track will produce the first 4 seconds of the main channel of the Title Screen theme, which you can hear below:

Link’s Awakening – Title Screen

Some opcodes are still not fully understood. Nonetheless, this is an incredible work, that allow us to both understand how the sound engine works, and how it can be modded to add new music tracks.

Transposition and trivia

Interestingly, only a handful of music tracks actually use a non-zero “default transpose factor”:

Except for the mini-boss, it seems that at some point during development the game designers decided to make musics playing in several houses higher-pitched. We can only guess why; but maybe that was to give a slightly more upbeat tone.

You can listen to the differences in MUSIC_HOUSE below:

HOUSE_MUSIC – Final pitch as in the released game.

HOUSE_MUSIC – Originally programmed pitch.

📦 Pre-composed save game

Did you know that Link’s Awakening includes a pre-composed saved game, that gives you right from the start all inventory items, dungeons keys, special items, and so on?

It was included as a developers feature, and disabled in the released game. But with a bit of tweaking, it’s easy to re-enable it. As Xkeeper and the fine people of The Cutting Room Floor documented more than ten years ago, enabling the Debug tools of the game will automatically write this pre-composed saved game to the first slot.

So Xkeeper dug into the code, and found how this feature works: under some conditions, a segment of data is simply copied to the save slot.

Before they documented this block of data, it looked like this:

label_4667::
    db 4, 1, 2, 3, 5, 6, 7, 8, 9, $A, $B, $C, 1, 1, 1, 0
    db 0, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2
    db 1, 1, 1, 1, 3, 1, 1, 1, 1, 4, 1, 1, 1, 1, 5, 1
    db 1, 1, 1, 6, 1, 1, 1, 1, 7, 1, 1, 1, 1, 8, 1, 1
    db 1, 1, 9

And after their work, and some bits of clever ASCII-art, the data for the pre-composed saved game now look like this:

DebugSaveFileData::
    db INVENTORY_SHIELD          ; B button
    db INVENTORY_SWORD           ; A button
    db INVENTORY_BOMBS           ; Inventory slots
    db INVENTORY_POWER_BRACELET  ; .
    db INVENTORY_BOW             ; .
    db INVENTORY_HOOKSHOT        ; .
    db INVENTORY_MAGIC_ROD       ; .
    db INVENTORY_PEGASUS_BOOTS   ; .
    db INVENTORY_OCARINA         ; .
    db INVENTORY_ROCS_FEATHER    ; .
    db INVENTORY_SHOVEL          ; .
    db INVENTORY_MAGIC_POWDER    ; .

    db 1  ; Have Flippers
    db 1  ; Have Medicine
    db 1  ; Trading item = Yoshi doll
    db 0  ; 0 Secret Seashells
    db 0  ; (@TODO "Medicine count: found?")
    db 1  ; Have Tail Key
    db 1  ; Have Angler Key
    db 1  ; Have Face Key
    db 1  ; Have Bird Key
    db 0  ; 0 Golden Leaves / no Slime Key

    ; Dungeon flags ...
    ;  +-------------- Map
    ;  |  +----------- Compass
    ;  |  |  +-------- Owl Beak / Stone Tablet
    ;  |  |  |  +----- Nightmare Key
    ;  |  |  |  |  +-- Small keys
    ;  |  |  |  |  |
    db 1, 1, 1, 1, 1 ; Tail Cave
    db 1, 1, 1, 1, 2 ; Bottle Grotto
    db 1, 1, 1, 1, 3 ; Key Cavern
    db 1, 1, 1, 1, 4 ; Angler's Tunnel
    db 1, 1, 1, 1, 5 ; Catfish's Maw
    db 1, 1, 1, 1, 6 ; Face Shrine
    db 1, 1, 1, 1, 7 ; Eagle's Tower
    db 1, 1, 1, 1, 8 ; Turtle Rock
    db 1, 1, 1, 1, 9 ; POI: unused? (9th dungeon?)

Way easier to understand.

As you can see, two details are a little curious:

🗺 Minimaps format

Although the Overworld map is accessible all the time by pressing the SELECT button, dungeon minimaps are displayed in the inventory.

Inventory displaying the Tail Cave minimap
The minimap of Tail Cave, the first dungeon.

To display those minimaps, the game stores one array of bytes per map. The 0xEF value stands for a simple room, 0xED for a room with a chest, and 0xEE for the dungeon Nightmare room.

In the disassembled code source, these maps were previously formatted as a simple array of values:

Minimap0::
    db   $7D, $7D, $7D, $7D, $7D, $7D, $7D, $7D
    db   $7D, $7D, $7D, $7D, $7D, $7D, $7D, $7D
    db   $7D, $7D, $7D, $7D, $7D, $7D, $EF, $7D
    db   $7D, $EF, $EF, $EF, $7D, $7D, $EE, $7D
    db   $ED, $7D, $EF, $ED, $EF, $ED, $EF, $7D
    db   $EF, $EF, $ED, $ED, $EF, $EF, $EF, $7D
    db   $EF, $7D, $EF, $ED, $ED, $7D, $7D, $7D
    db   $7D, $ED, $EF, $EF, $7D, $7D, $7D, $7D

Minimap1::
    db   $7D, $7D, $7D, $7D, $7D, $7D, $7D, $7D
    db   $7D, $ED, $ED, $ED, $EF, $EF, $EF, $7D
    db   $7D, $7D, $ED, $7D, $7D, $ED, $7D, $7D
    db   $7D, $EF, $EF, $7D, $7D, $EF, $EE, $7D
    db   $7D, $EF, $7D, $7D, $7D, $7D, $EF, $7D
    db   $7D, $ED, $7D, $7D, $7D, $7D, $EF, $7D
    db   $7D, $EF, $EF, $EF, $EF, $EF, $EF, $7D
    db   $7D, $7D, $ED, $ED, $ED, $ED, $7D, $7D

Xkeeper found a clever way to make this data more readable: using rgbasm charmaps.

In the source code, a CHARMAP command tells the assembler how to convert ASCII characters to sequences of bytes. This allow for instance to map the text of dialogs to the indices of the tile to use for each letter. Xkeeper found these charmaps another use: by defining a custom charmap, it becomes possible to format the dungeon minimap data as text and symbols.

NEWCHARMAP MinimapCharmap
CHARMAP "  ", $7D   ; Blank (not shown on map)
CHARMAP "##", $EF   ; Room (shows up on map)
CHARMAP "Ch", $ED   ; Room with chest
CHARMAP "Nm", $EE   ; Nightmare boss marker

At compile-time, the charmap gets the text converted to the expected bytes. Which means that the dungeon minimap data now looks like this:

    ;    0 1 2 3 4 5 6 7  - Minimap arrow positions.
Minimap0::
    db "                "
    db "                "
    db "            ##  "
    db "  ######    Nm  "
    db "Ch  ##Ch##Ch##  "
    db "####ChCh######  "
    db "##  ##ChCh      "
    db "  Ch####        "

Minimap1::
    db "                "
    db "  ChChCh######  "
    db "    Ch    Ch    "
    db "  ####    ##Nm  "
    db "  ##        ##  "
    db "  Ch        ##  "
    db "  ############  "
    db "    ChChChCh    "

Much more readable, and easier to edit.

Plus we can now clearly see the map of the first dungeon (Tail Cave) being shaped like a Mini-Moldorm, and the second dungeon (Bottle Grotto) being shaped like the jar of the boss.

What’s next?

With source code shiftability achieved, the next point of focus is graphics data. For now the graphics of the game are not so easy to edit: many of them are laid out in a complicated way, using baroque color palettes. Some issues have been opened: hopefully we’ll find a way to convert all graphics to easily editable sprite sheets, that can be transposed to the format expected by the engine at compile time.

Marijn is also close to merge an impressive PR that allows to build every single revision and language of the game.

And of course, the documentation of the physics engine and entities behavior is still an ongoing work.

Want to read more? Discover more of the code, or join the discussion on Discord.


Notes

  1. You may already know Drenn, because years ago he gave a new start to this project. He dived into all the numerous errors that had crept in the then-partial disassembled source code, and fixed them all. Moreover, he added a checksum step, to ensure the code would never diverge from the compiled game again.

    Drenn is also working on a fairly complete disassembly of Zelda Oracle of Ages/Oracle of Seasons–including a level editor for these games. Check it out! 

Certaines personnes

Ce texte est une traduction de « Some people », de Jason Kottke.

Certaines personnes se sentent angoissées et impuissantes.

Certaines personnes s’ennuient.

Certaines personnes sont isolées en confinement et se sentent seules.

Certaines personnes se rendent comptent que l’Après sera très différent de l’Avant.

Certaines personnes profitent du temps supplémentaire avec leurs enfants et quand ça sera fini, ce temps leur manquera.

Certaines personnes viennent de sortir de leur douzième créneau d’affilé à l’hôpital et ne peuvent pas embrasser leur famille.

Certaines personnes ont mangé à leur restaurant préféré pour la dernière fois et ne le savent pas encore.

Certaines personnes sont mortes du coronavirus.

Certaines personnes ne peuvent pas s’empêcher de lire les nouvelles.

Certaines personnes n’ont pas les moyens d’acheter du savon.

Certaines personnes apprennent à faire du pain.

Certaines personnes travaillent de chez elles tout en essayant de faire l’école à la maison.

Certaines personnes sont des parents seuls qui essayent de travailler de chez eux tout en essayant de faire l’école à la maison.

Certaines personnes ont du mal à boucler les fins de mois et la prochaine paye ne viendra pas.

Certaines personnes sont inaptes à la fonction de président.

Certaines personnes ont quitté la ville pour leurs résidences secondaires.

Certaines personnes ne peuvent pas faire les courses parce qu’elles sont des personnes à risque.

Certaines personnes ont perdu leur travail.

Certaines personnes n’arrivent pas à dormir.

Certaines personnes regardent gratuitement des opéras en ligne.

Certaines personnes sont en quarantaine depuis plusieurs semaines.

Certaines personnes ne peuvent pas télé-travailler.

Certaines personnes ont attrapé le coronavirus et ne le savent pas encore.

Certaines personnes sont trop angoissées pour se concentrer sur leur travail.

Certaines personnes n’ont pas les moyens de payer leur loyer du mois prochain.

Certaines personnes continuent à se réunir en grands groupes.

Certaines personnes prennent de vrais risques pour sauver nos vies.

Certaines personnes n’ont pas acheté assez de solution hydro-alcoolique.

Certaines personnes ont acheté trop de solution hydro-alcoolique.

Certaines personnes n’ont plus accès à leur thérapeute.

Certaines personnes ne peuvent pas aller travailler mais sont toujours payées par leur employeur. Pour l’instant.

Pour certaines personnes, le principal souci est de décider ce qu’elles vont regarder ensuite sur Netflix.

Certaines personnes se portent volontaires.

Certaines personnes ont une entreprise qui va faire faillite.

Certaines personnes se rendent comptent que les enseignants sont formidables.

Certaines personnes commandent à emporter aux restaurants du coin.

Certaines personnes voudraient vraiment juste un câlin.

Certaines personnes n’arrivent pas à convaincre leurs parents âgés de prendre tout ça au sérieux.

Certaines personnes s’inquiètent de leurs investissements en bourse pour leurs vieux jours.

Et certaines personnes n’ont jamais eu d’investissements.

Certaines personnes vont être confrontées à plus de violences domestiques.

Certaines personnes vont tomber malade ou se blesser et auront plus de mal à accéder à des soins médicaux.

Certaines personnes ne peuvent pas acheter la nourriture dont elles ont besoin parce que les produits acceptés par les aides sociales alimentaires ne sont plus en rayon.

Certaines personnes ne veulent pas arrêter de faire la fête.

Certaines personnes ont perdu leur solution de garde d’enfant.

Certaines personnes font tout ce qu’elles peuvent pour rester calmes et garder l’espoir et ça ne marche pas.

Certaines personnes regardent Contagion et jouent à Pandémie.

Certaines personnes ne savent pas ce qu’elles vont faire.

Certaines personnes sont surchargées de conseils sur comment travailler depuis chez soi.

Certaines personnes mangent ou boivent trop.

Certaines personnes pensent à l’après.

Certaines personnes sont contrariées de ne pas pouvoir voyager.

Certaines personnes sont en manque de sexe.

Certaines personnes prévoient de jardiner plus cette année.

Certaines personnes ne vont pas voir leur famille pendant des mois.

Certaines personnes se déconnectent pour garder la tête froide.

Certaines personnes n’arrivent pas à voir le bout du tunnel.

Certaines personnes vont se rendre compte qu’il faut qu’elles rompent avec leur conjoint.

Certaines personnes chantent Imagine all the people.

Certaines personnes ne sont pas dans cette liste.

Ces expériences sont toutes celles de vraies personnes, tirées de journaux, des réseaux sociaux, et d’amis. Courage : vous n’êtes pas la seule personne à passer par ce que vous vivez en ce moment. Mais soyez attentionné : tout le monde ne passe pas par la même chose que vous. Même si en dernier lieu, tous nous sommes affectés par ces mêmes évènements.

Jason Kottke