3

A Mathematician’s Guide to Wordle

 2 years ago
source link: https://aperiodical.com/2022/02/a-mathematicians-guide-to-wordle/?ref=sidebar
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client

You're reading: Irregulars

A Mathematician’s Guide to Wordle



By Ali Lloyd. Posted February 1, 2022 in Irregulars



We invited mathematician and wordplay fan Ali Lloyd to share his thoughts on hit internet word game phenomenon Wordle. If you’re not familiar with the game, we recommend you go and have a play first.

Photo of a mastermind board, showing coloured pegs in rows.CC BY-SA ZeroOne

When I first saw Wordle I said what I saw many other people subsequently say: “Oh, so it’s a bit like Mastermind but with words? That’s a neat idea”. 

In formal language terms, Mastermind is played over an alphabet of six ‘symbols’ (represented by six differently coloured pegs) and a ‘word’ is any combination of those symbols of length 4. There are therefore 64=1296 different words, all of which are well-formed – any combination of 4 symbols is a valid word.

In a game of Mastermind, a target pattern is fixed, and the player takes guesses by placing pegs in the board. Red and white pegs are used in response to indicate which colours in the guess are in the right place and which are present in the target pattern, but in a different place. The player then takes another guess. If you’re playing Consistent Mastermind, the next guess must incorporate all the clues given after previous guesses. The amount of allowed guesses before you lose is typically between 8 and 12.

The fact that all words are well-formed simplifies Mastermind strategy somewhat: it makes it completely uniform, in the sense that all combinations of colours are valid guesses and the strategy itself does not need to change depending on which colour and position you have information about.

As an illustration of this, Donald Knuth has an algorithm for Mastermind which is guaranteed to win in 5 guesses or fewer, and involves starting with a combination like ‘red red blue blue’ – however, this could equally be ‘yellow yellow green green’ or ‘red blue red blue’; the important thing is that there are two of each.

Wordle

Wordle is very similar to Mastermind, except it uses the alphabet as the ‘alphabet’, and English words as the ‘words’, and you only get 6 guesses. It also has a consistent version, where you have to use the previous clues marked as correct in subsequent guesses – in Wordle this is called ‘Hard mode’. Hard mode Wordle is arguably harder than Consistent Mastermind as your guess has to fit the pattern and also be a valid word. On the other hand, this pushes you to solve more directly.

Plainly the uniformity of Mastermind does not exist in Wordle – ARERE ((Spenser) backward, behind) is probably a better opening guess than XYLYL (a chemical compound found in coal-tar). Indeed, while it would be easier to come up with an algorithm for solving anything’s-a-word Wordle, it would not be much fun at all to actually play.

Wordle has both a ‘source’ word list and a ‘target’ word list. You can guess anything from the source word list (which is the 5 letter words from CSW19). Incidentally there is exactly one 5-letter word in the US Scrabble dictionary that does not appear in CSW19 – so bad luck if you really want to guess LUVED.

The target word list is a small hand-curated subset of less than 2500 of these words. This complicates potential strategy in two ways: firstly, it involves the consideration of whether the word has been deemed common enough to be fair by a specific other person; secondly, it opens up the possibility of guessing a word which will definitely not be correct, but will rule out enough to make it worthwhile. 

One of the reasons a curated list is a good idea is that CSW19 is chock-full of potentially horrible traps, especially for hard-mode. If the target word is HILLS and you are unlucky enough to guess BILLS on your first go, you end up with B I L L S with any one of (B)CDFGHJKLMNPRSTVWYZ the correct first letter.

As it happens, in fact none of these words actually appears in the Wordle target word list, as most words ending in S did not make the cut. However the second-worst scenario of this kind in CSW19, ? I G H T, which has 15 possible starting letters (ABDEFHKLMNPRSTW), is still a potential Wordle nightmare, with 9 of those being included in the target word list

Another theoretical pitfall is repeatedly getting yellows instead of greens. In mathematical terms, the worst-case scenario here would be akin to a cyclic permutation of length 5 (i.e. with no fixed points) – thankfully there is no such cycle in CSW19, but it comes close – PESTO, ESTOP (to hinder or preclude), STOPE (to excavate in layers) and TOPES (drinks liquor to excess) are all in there, meaning it’s theoretically possible to have three straight rows of yellows (of course if that did happen you would probably get the answer on the next go):

T O P E S
S T O P E
E S T O P
P E S T O

Frequency Analysis

The fact that not all letters are equally likely to occur in the answer word can be used to your advantage when guessing. Everybody will likely be doing this already, knowing intrinsically that RATES is more likely to give you information about the answer word than RAZES. Of course if you get lucky and there is a Z, fair play to you. But in all likelihood you will discover that there isn’t a z, which doesn’t narrow down your options much. On the other hand, all three bits of information about that T are reasonably useful – if it goes green, you narrow the list of possible words down to 616. If yellow, 2417. If black, take consolation from the fact you’ve eliminated up to 3000 words from the list.

It is well known that when you order letters by frequency of appearance in English words, you get ETAOIN SHRDLU. This is especially well known to anyone who has solved a substitution cipher, or written a substitution cipher solver – for ciphertext of a reasonable length, where the original has not been constrained deliberately to make decoding harder, starting out by mapping letters by frequency in ETAOIN SHRDLU order is almost always close enough to the plain text that it only requires a few swaps to get there.

However, this is highly dependent on the word list you are analysing. If you take all 5 letter words in CSW19, you get SEAORY LTNUDY. If you instead take the 5 letter words in Google’s 10,000 common words list, you get ESAROT LINDCH.

If you instead take the words that have already been Wordle answers, you get ERAOTI LSCNUD. You can even dig into the source (not very deeply) and find the complete list of words that will be Wordle answers (this thread discussing this does not contain spoilers, unless you consider the letter distribution of the answers a spoiler).

Using this letter distribution has apparently yielded results for more than one person. The unusually high prevalence of the letter C and low prevalence of H can inform your strategy – if you’re thinking of trying SHRUB, maybe consider SCRUB instead.

Forget “Etaoin shrdlu”. For the 2315 words used in Wordle, the new coolness is: “Earot Lisncuyd”.

The full frequency table is:

e:53% a:42% r:39% o:33% t:31% l:31% i:29% s:29% n:25% c:21% u:20% y:18% d:17% h:17% p:16% m:14% g:13% b:12% f:10% k:9% w:8% v:7% z:2% x:2% q:1% j:1%

— Paul Lamere (@plamere) January 4, 2022

Frank Swain points out that it probably isn’t just a case of how frequent letter occurrence is, nor even just the frequencies for each word position, but how often letters appear together, and has produced this heat map:

Image

This uses a corpus of 5757 words which I assume is Donald Knuth’s 5757 list (him again!) – a list which is often used as the basis for programming challenges.

Taking that idea and running with it gives you the basis of a potential strategy – try and guess things which are likely to appear in the same word – i.e. attempt to maximise positive information about the answer word. One way to do this is take every guessable word still in consideration and figure out how much information it gives you about the answer, for each of the other words in the list. 

Solver

I wrote a (hard-mode) solver along the above lines, and it does ok. There are many potential refinements that could be made – in particular it does nothing to avoid potential ? I G H T situations, and could probably do with a more carefully chosen initial word. The solver looks at all the remaining possibilities and for each, computes the sums of the number of greens and yellows that word would give if you guessed it, for all possible answer words.

It has four options: it either uses a 3/2 green/yellow weighting to build an overall score for each word based on the remaining possibilities, or it is greedy for greens – ordering lexicographically <# greens, # yellows>. It can either favour common words or not. If it favours common words, it promotes any words from Google’s 20,000 common words list to the top, if they are already in the top 20 options for next word.

The initial word picked by each strategy is as follows:

  • green-greedy strategy that doesn’t favour common words – SORES
  • green-greedy strategy that favours common words – CORES
  • weighted strategy that doesn’t favour common words – TARES
  • weighted strategy that does favour common words – CARES

Here is how each performs over the first 224 Wordle puzzles:

And here is how they stack up in terms of win percentage and average guesses:

Max GMax G + favour commonWeighted G/YWeighted G/Y + favour commonWin %84.82%95.09%89.29%94.64%Average guesses4.5894736844.0563380284.3753.900943396

You can experiment with my solver (christened Solvador D’Ali by a friend), and a ‘next word suggester’ I also made.

One thing about all of these strategies is that they don’t include any meta-information about previous Wordles. For example, it could be made to ‘notice’ that no previous Wordle has been a plural, and add in a weighting against selecting plurals.

As an alternative to trying to minimize your average number of guesses, you might want to just maintain your streak. Friend of the Aperiodical Andrew Taylor has implemented a strategy (where he summarises the first guess as “how few solutions would give the same clues in the worst case”) which always wins. It’s essentially a proof-by-cases, and each case keeps going until the list of possible word is less than or equal to the number of guesses remaining. There is also this ‘easy mode’ strategy of ruling out as many letters as possible.

So even though you know the answer probably won’t be EEVEN (the latter part of daylight hours) or VOZHD (a supreme leader in Russia), whichever way you look at it, strategy is still just a numbers game. The question is, which numbers are important?

Ultimately, it seems very likely that some sort of ‘genetic algorithm’ would be the best Wordle solver, just like in Mastermind. Solving a Mastermind board is an NP-complete problem. If you’re interested in that, maybe you would enjoy playing Gödle instead?

Starting word

As we’ve seen above, TARES, SORES, CARES, CORES, REAIS, BLAHS, SOLAR and many others potentially make good starting words. If you have a hard time deciding why not play ‘Wordle Legacy’?

Or if you fancy a real challenge, here is a list of some of the most useless starting words:

ZOPPO (having a syncopated or temporarily changing accent of a beat)
PHPHT (used as an expression of annoyance)
KIBBI (an Eastern dish of ground lamb and crushed wheat)
WHIZZ
QAJAQ (Inuit word for a kayak)
OXBOW
FLUFF
YUKKY
KUDZU (an ornamental papilionaceous plant of China and Japan)
FUZZY
JUGUM (a pair of opposite leaves)
FUFFY (Scots for light and soft)
HYPHY (a style of hip-hop music originating in the Bay Area of San Francisco)
IMMIX (archaic to mix in, commingle)
XYLYL

Sharing

With the news of Wordle’s buyout by the New York Times, you might wonder what it is they have actually bought. After all the concept of the game long predates Wordle’s existence, notably in the game show Lingo.

Wordle’s primary innovation, which ensured it became a viral sensation, is its shareability. This arises from the communal effect of having a fixed daily word (so that everyone is doing the same puzzle), and the neat method of communicating your result via spoiler-free sequences of coloured squares. The shareable string of squares and Twitter were practically made for each other.

As well as people sharing their results every day, this leads to all sorts of Twitter shenanigans. The account WordleStats trawls Twitter looking for Wordle results and summarising them. There have been Pokémon references, and recreations of the loss meme; even Kyle MacLachlan getting a Twin Peaks reference in; you can make a town out of your pattern of squares, or you can do something else entirely.

Incidentally, the unicode codepoints used for the sharing string are: White large square (U+2B1C) or Black square (U+25A0), Large yellow square (U+1F7E8) and Large green square (U+1F7E9). These last two are rather excitingly (if you like that sort of thing) in the second plane of unicode characters, the Supplementary Multilingual Plane (SMP), and they are codepoints consisting of two UTF-16 codeunits.

One interesting consequence of this is that the exact format of Wordle shareable results wouldn’t have been possible (or at least widespread) a few years back where support for display of SMP characters was nowhere near universal!

Variants

Some other variants you may be interested in:

My daily regulars are Wordle, Lewdle and Facle, a Scottish Gaelic version of Wordle.


About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK