Abstract
Increases in the complexity of the JN-25 cipher systems from 1939 to 1943 must have suggested that developing new techniques in anticipation of future challenges was desirable. A section of Op-20-G devoted to Mathematical Analysis and Machines and called Op-20-GM was established early in 1942 and must have had the JN-25 systems as a high priority. A peculiarity found in the analysis of the 33,334 book groups underlying these systems turned out to yield important new methods of searching for alignments and did not depend on the breaking of indicator systems. This chapter explains these developments.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The Simpson report is in the British Archives as item HW 8/149. It should not be confused with the technical report by J. W. S. Cassels and Simpson available as item HW 43/34. This latter was declassified only in 2013.
The link with Marshall Hall is confirmed by the record of a presentation on the scanning distribution given by Hall in the Dayton Navy Computation Laboratory on 18 December 1943.
- 2.
In summary, the use of only scanning groups in most JN-25 code books was a major blunder.
Those who like that sort of thing may wish to read David Wragg’s Snatching Defeat from the Jaws of Victory for an account of 28 of the biggest military blunders of the twentieth century, starting with the 1905 Battle of Tsushima. Other such books are Saul David’s Military Blunders: The How and Why of Military Failure, Robinson, London, 1988 and Kenneth Macksey’s Military Errors of WW2, Cassell, 2005. These books do not mention JN-25.
- 3.
This has already been quoted in Appendix 3 of Chap. 10.
- 4.
Hall’s autobiographical notes are on pages 367–374 of Peter Duran, Richard Askey and Uta Merzbach A Century of Mathematics in America, volume 1, American Mathematical Society, Providence, RI, 1989.
- 5.
There is a variant of the Mamba method for putting JN-25 GATs into depth when there are several active operators who are reliable tailers. It is assumed that each of them chose a starting point in a new additive table independently and worked on from there. The indicator system in use may well be totally incomprehensible. The practice of tailing would produce long stretches of GATs encrypted with consecutive additives from the unknown table and one could use this method on, say, 100–200 such GATs to find possible alignments with another long stretch. The challenge of mechanising the attack on such a system is not considered here at all.
- 6.
This is essentially the convolution square of the scanning distribution.
- 7.
The reference is NARA RG38 CNSG Library Box 15 item 3222/54. This was discovered by Ralph Erskine of Belfast, who is most heartily thanked for drawing it to our attention. The document is anonymous and carries no date but presumably was written in the second half of 1943. Another document in RG457 Box 622, item 1682, dated November 1943 and entitled Research Report on Alignment of JN-25 Traffic is also of interest.
The counts of the numbers of multiples of 3 with the various characteristics may be found in NARA but have been checked using a modern computer.
NARA RG457 Box 705, item 1742 includes copies of a report by LtCdr E. W. Knepper and Lt (Jg) L. E. Shinn entitled A Proposed Method for Placement of Partially Keyed Messages in Additive Systems when some Additives are Available. This appears to have been written in March 1944 or perhaps a little earlier. The document appears to refer to situations in which some progress has been made by a baby brute force procedure coupled with the methods set out in Chap. 12 to recover a run of additives from a page.
Presumably Knepper was the principal inventor of knepperizing, a third method of decrypting JN-25 systems once the common book groups are known. This is described in the GYP-1 Bible. More information is given in History of GYP-1, NARA RG38, CNSG 5750/202.
Other interesting references to the technology of the era are in RG457 Box 705, items 1741 and 1743 on Tape scanner for double repeats, Japanese JN-25 traffic, 1943–1945.
- 8.
Although the IJN operators added the ‘additives’ to the book groups to obtain the GATs and subtracted it from the GATs to recover the book groups, Op-20-G found it convenient to use the opposite practice after it had reconstructed the encryption process. Its version of the additive table was the negative of the IJN original. Thus Op-20-G decryption was carried out by (false) addition. This ‘Op-20-G usage’ is used in this chapter here for the sole purpose of making the surviving Mamba cards in NARA RG457, Box 705, slightly more intelligible. See Note 19 of Chap. 8.
The section entitled A Distinction in Additive Table Usage in Sect. 8.14 explained that the IJA and IJN used different practices when at the end of a page of additive. The Army usage was to go back to the first additive on the same page, while the Navy went on to the first additive on the next page. In effect the IJA was using numerous short tables of additives, while the IJN used only one long table.
- 9.
As noted earlier, large-scale mental arithmetic is tiring and unreliable. Automation is most desirable for problems like this.
- 10.
See Appendix 2 of Chap. 10.
- 11.
The choice of starting place that yields the next lowest number of these digits turns out to be the 870th, with the first group used turning out to be 27330. This produces 11 occurrences of 2, 3, 6 and 9. Various experiments with similarly produced random data all found the starting place, sometimes just as one of two or three possibilities. Thus Mamba seems to work in practice. It would be less reliable for shorter messages, say with around 30 GATs.
- 12.
The IBM card (see Sect. 1.15) was made of firm thin quality cardboard of standard size. Holes could be punched in at the intersection of any of its 12 rows and 80 columns. These holes were punched with considerable precision by a card-punch.
NARA RG457 Box 705, item 1742 includes an envelope marked MAMBA and dated 2 May 1944. The contents are described as ‘Samples of maximal and minimal cards, punched on an I.B.M. printing punch modified by (illegible) for Op-20-GY-P.’ The initials ‘J. H. H.’ are those of Lieutenant John H. Howard of Op-20-G who was heavily involved in this project. The ‘X’ (top) rectangles have all been punched out: this reflects the practice (NARA RG38 CNSG Library Box 15, item 3222/54 page 2) ‘A measure of the overlap at any given alignment can be obtained by brushes reading the number of overlap holes in common between the additive card and message card’. The ‘X’ and ‘Y’ rows are not shown in the displays representing cards in this chapter.
These cards must be among the most remarkable relics of WW2 cryptology in NARA. They may be compared with the ‘Banburismus’ used at Bletchley Park against Enigma. It also was based on alignment of holes punched in pieces of paper.
- 13.
NARA RG38 CNSG Library Box 15 item 3222/54 also contains minutes of a meeting held at Op-20-GYP on 12 April 1944. Page 3 makes explicit mention of the use of stencils in the Mamba process.
The reader may wish to compare this process with the ‘Banburismus’ used at Bletchley Park against Enigma.
- 14.
NARA RG457 Box 705, item 1742. This indirectly reveals the magnitude of the dependence upon the use of multiples of three as book groups in JN-25. Note 2 is relevant here: the Mamba method would be much less useful against the IJA practice of using numerous short tables of additives.
- 15.
The machine was made in the NCR factory in Dayton, Ohio. This had been visited by Alan Turing in December 1942. The general story is described by Jim DeBrosse and Colin Burke in The Secret in Building 26, Random House, NY, 2004, which, however, does not mention Mamba. Considerable material on the Mamba project and various other such projects may be found in the RG38 and RG457 sections of the College Park NARA. Thus in RG457 there are Box 0584, item 1445 Operations and maintenance schedule for Mamba; Box 0591, item 1494 Brief descriptions of RAM equipment; Box 0616, item 1653 Mamba (Scanner schematics); Box 0705, item 1742 Mamba, 1944–1945; Box 0949, item 2799 Cryptanalytic machines synopsis, 1944; Box 0950, item 2807 Use of high-speed crypto equipment; Boxes 1070 and 1071, item 3390 NCR schematics and unidentified machine drawings. There is also much material on the Copperhead I project.
The Dayton code breakers website includes a photograph of the Mamba machine. It is taken from RG457, Box 804.
- 16.
The symmetry or functional equation \(p(s)\ =\ p(5 - s)\) implies that five of the Fourier coefficients are zero.
The reader may wish to repeat these calculations for the function q of the ‘weakness 2’ section where q(0) = . 13051, \(q(1)\ =\ q(9)\ =.09056\), etc. This has the symmetry or functional equation \(q(s)\ =\ q(-s)\) which implies that the coefficients of the five sine terms (f 5, f 6, f 7, f 8, f 9) are zero.
- 17.
In an era before pocket calculators were introduced some rounded numerical values would have been used.
- 18.
The strategy is to choose a GAT that decrypts to a book group with characteristic 6, 9, 2 or 3 and there are then only 925, 925, 1780 or 1780 potential decryptions respectively.
- 19.
It is possible that the characteristic may help find the additive in a Phase 2 decipherment attempt that is not yielding a result quickly. It may well be possible to use a mixture of difference and Fourier methods to assist in the decryption of around 20 JN-25 GATs in depth when nothing is known about the more common book groups. However it would appear that in practice working out the optimal strategy would need modern electronic calculation capacity and so would be anachronistic.
- 20.
Part F of the CBTR, apparently due to Professor Room, refers to special roles played by the prime number 11 in the construction of certain Japanese code books. According to the CBTR at least some early commercial codes had the feature that the more common words and phrases were encoded by numbers that were multiples of 11. This has not been verified and may be an error.
Author information
Authors and Affiliations
Appendices
Appendix 1 Other Characteristics
Consider an additive system using 4-digit book groups. As an error-correcting device each 4-digit group abcd could be extended to a five-digit group abcde where e is chosen so that the false sum \(a + b + c + d + e\) is 0. For example 5294would be extended to 52940. The extended book groups are then encrypted by means of five-digit additives. Extending book groups in this way turns out to be insecure because it betrays alignment.
Earlier in this chapter, the characteristic of a 5-digit group abcde was defined to be the (false) sum \(a + b + c + d + e\). The following general algebraic relationship holds: the characteristic of the (false) sum of two groups is just the (false) sum of the characteristics of the two groups. Thus if 4-digit book groups are extended to become 5-digit groups with characteristic 0, the characteristics of the GATs obtained by additive encryption are the same as the characteristics of the corresponding additives. For example if the message consists of the eight book groups shown and the additives are as given on the third line:
Message | 2360 | 6797 | 7499 | 7896 | 9640 | 9173 | 6687 | 3127 |
Extended | 23609 | 67971 | 74991 | 78960 | 96401 | 91730 | 66873 | 31277 |
Additive | 64575 | 13110 | 64590 | 59050 | 16157 | 53639 | 26042 | 57102 |
Characteristic | 7 | 6 | 4 | 9 | 0 | 6 | 4 | 5 |
False sum = GAT | 87174 | 70081 | 38481 | 27910 | 02558 | 44369 | 82815 | 88379 |
Characteristic | 7 | 6 | 4 | 9 | 0 | 6 | 4 | 5 |
Here the characteristics may be calculated by the other side and used to put intercepts into depth. Suppose two messages in this code are being examined. The characteristics are taken to be 3-6-0-6-7-9-7-7-4-9-9-7-8-9-6-9-6 and 8-9-6-9-6-4-0-9-1-7-3-6-6-8-7-3-1-2 respectively. These two strings of digits have the segment 8-9-6-9-6 in common and so it is extremely likely that the correct alignment has the last five groups of the first over the first five groups of the second. That is all the leakage of information but it is quite serious enough. This simple redundant encryption allows the enemy to bypass the first defensive barrier (the indicators) of an additive system.
If, instead, the extra digit is inserted after encryption then there is no loss of security—nothing is given away. (Indeed, nothing would be given away if the same encrypted message were transmitted twice.) The calculations become:
Message | 2360 | 6797 | 7499 | 7896 | 9640 | 9173 | 6687 | 3127 |
Additive | 6457 | 1311 | 6459 | 5905 | 1615 | 5363 | 2604 | 5710 |
False sum | 8717 | 7008 | 3848 | 2791 | 0255 | 4436 | 8281 | 8837 |
Extended = GAT | 87177 | 70085 | 38487 | 27911 | 02558 | 44363 | 82811 | 88374 |
Characteristic | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
A different characteristic was mentioned in Sect. 14.5. An indicator group abcd was constructed so that the false sum/difference \(a + b + c - d\) was always 0. This was exploited. To return to book groups in additive cipher systems, almost any limitation on their choice is likely to be detected by the other side and quite possibly exploited to weaken security. Random choice of book groups is much safer.
Appendix 2 A Distribution for JN-11A
As noted before, JN-11 and JN-25 evolved in quite different ways in 1943 and 1944. Thus there is no historical interest in the observation that there would be an analogous process for an additive system in which the code book used 4-digit groups such that each book group had digits that summed to one more than a multiple of three. One can calculate the remainders upon division by 10 of each of the 3333 such book groups and confirm that 282 have remainder 0, 60 have remainder 1, 540 have remainder 2, 480 have remainder 3, 45 have remainder 4, 348 have remainder 5, 633 have remainder 6, 120 have remainder 7, 165 have remainder 8 and 660 have remainder 9. A Mamba process can be concocted for this context. The ‘minimal’ digits relative to modulo 10 false sums of JN-11A book groups are 1, 4, 7 and 8.
Appendix 3 Mamba-11
There are other possible characteristics. For example \(a - b + c - d + e\) or \(a + b + c + d - e\) could be taken to be ‘the’ characteristic and the additive system could be designed with this taking some fixed value on all code groups. Once the cryptanalysts detected such a practice was current, the above method of putting GATs into depth (alignment) would be available.
This chapter has dealt with a technique used to attack additive ciphers whose book groups were all multiples of 3. Suppose instead that all the book groups of the additive cipher being targeted were all multiples of 11. We recall that a 5-digit number abcde is a multiple of 11 if the alternating sum \(a - b + c - d + e\) is a multiple of 11. This alternating sum may well be negative. Thus for the number 29183 the alternating sum is \(2 - 9 + 1 - 8 + 3 = -11\) and so 29,183 is a multiple of 11. One may check that 29, 183 = 2653 × 11.
Then there is an analogue of the above processes available which may as well be called Mamba-11. It is presented here for 4-digit groups.
It is inappropriate to work with the characteristic \(a + b + c + d\) of the digits of the group abcd in the Mamba-11 context. Instead one works with the false alternating sum \(a - b + c - d\). Then Mamba-11 works similarly to the original Mamba process. If instead (say) the book groups were chosen so as to leave remainders of 0, 5 or 9 upon division by 11, a modified Mamba-11 process would be available.
There might have been a Mamba-11 process available for an additive cipher system using 4-digit groups for which the book groups of an excessive proportion of the commonly used words were multiples of 11. Despite the comments on this matter in Parts FFootnote 20 and G of the CBTR, the authors know of no documentation of such a use of Mamba-11.
Appendix 4 Thwarting All This
In theory the IJN codemakers could have beaten all four of the methods given for exploiting the scanning distribution. All that was needed was choosing randomly 925 groups of each characteristic and then using the 9250 groups so obtained randomly as book groups for the 9250 most common words in new JN-25 code books. In practice any understanding of why this was necessary would have convinced any communications security unit that requiring all book groups to be scanning was a pernicious practice. So it is safe to assume that JN-25 book groups were chosen randomly, with about 2.7 % having characteristic 9, etc.
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Donovan, P., Mack, J. (2014). The Scanning Distribution. In: Code Breaking in the Pacific. Springer, Cham. https://doi.org/10.1007/978-3-319-08278-3_15
Download citation
DOI: https://doi.org/10.1007/978-3-319-08278-3_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-08277-6
Online ISBN: 978-3-319-08278-3
eBook Packages: Computer ScienceComputer Science (R0)