GPT-2 Style Pre-tokenisation

Before BPE runs, GPT-style tokenisers split text into chunks with a regex, so merges never cross word boundaries. GPT-2's pattern (simplified to ASCII here) keeps a leading space attached to the word:

• contractions: 's, 't, 're, 've, 'm, 'll, 'd
• ? + a run of letters
• ? + a run of digits
• ? + a run of other characters that aren't letters, digits or whitespace
• runs of whitespace not followed by a non-space (so that the last space attaches to the next word), and any remaining whitespace

Implement pretokenize(text) with re.findall and this pattern:

's|'t|'re|'ve|'m|'ll|'d| ?[A-Za-z]+| ?[0-9]+| ?[^\sA-Za-z0-9]+|\s+(?!\S)|\s+

Then implement count_pretokens(text) and longest_pretoken(text), the first-occurring longest chunk. Joining the pre-tokens must always give back the original text.