Group the bytes before processing

A patch boundary cuts through 猫

Botie can encode unfamiliar text as bytes, but one step per byte can create a long sequence. Grouping neighbouring bytes shortens the sequence without requiring word entries.

BOTIE

The text 猫a contains a three-byte character followed by one ASCII byte. Must each two-byte patch decode as a character by itself?

The inputs we are comparing

Input text

Example 1

bluecats

Example 3

猫a

Result for example 1

98	108
117	101
99	97
116	115

Choose an explanation to test

Yes, every patch must coincide with one visible characterNo, fixed-width patches can split a character's encoding

A multibyte character reaches a short final patch

This check uses the pieces from “Group the UTF-8 stream into byte pairs”.

BOTIE

Encode Aé as UTF-8 and split the byte sequence into consecutive patches of at most two bytes. Return every byte, including the final shorter patch. Do not realign patches to character boundaries.

Input text: Aé

What result should your machine produce for this case?

Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.

Build a construction in the workspace above first. This case will test those pieces.

What must be preserved so the complete byte stream can be reconstructed from the patches?

Your notes for later. These stay with this journey.

Grouping can preserve all bytes while changing the units and boundaries processed by a model.

This is a small construction problem. The lesson explains the assumptions and the parts of the full method that this example leaves out.

Work through the Python challenges →

Continue the courseCodebook Quantisation →

← Hashing Characters