sci.crypt Sandbox - Scoring
Professional mode (go to all ciphers)

Speed (Sp)

The speed relative to a particular implementation of a known cipher, on a particular architecture. The "standard" implementation is the "fast" version of the code submitted for AES, and the architecture is a AMD Barton 2500@15x140=2100MHz (or approximately about XP2800 rating). This benchmarks at 0.001497 seconds per 64kb file, which comes out to 41.75 MBytes/sec. This figure may change. The speed of a submitted cipher is also taken for a 64kb file by adding one key setup to the time taken to encrypt 64kb. The relative speed is then transformed to a score through the inverse tangent function: 
  Sp = atan((P/41.75)^(1/4))*2/pi
  where
    P = speed of algorithm in mbytes/sec
    pi = 3.141592654...
This clearly gives a score between 0 and 1.

Brokenness levels (Sb)

LevelScore
No attacks0% (Sb = 0)
Distinguisher10% (Sb = 0.1)
Chosen plaintext30% (Sb = 0.3)
Known plaintext60% (Sb = 0.6)
Known ciphertext100% (Sb = 1.0)

Each level then multiplied by a value which I call the break multiplier, which rates how "good" the break is:
  M = (1-K*(lg(T)^2 + lg(S)^2))/C
  where:
    T = time for attack
    K = 1/(B^2)
    B = bits per key
    S = space for attack (bytes)
    C = "Minimum security" constant
The space required for an attack includes any plaintext or ciphertext that is required.

The minimum security constant decides when a cipher is regarded as completely broken. The lower the constant, the higher the requirements for a cipher would be. Currently set to 0.875 which rates an attack as perfect if it requires 2^32 time and 2^32 space when the cipher uses a 128 bit key. Note that this means that M can go above 1 if the attack is very good, which also means that a cipher's score can go below zero.

Take a cipher with a 128-bit key, which has an attack taking 2^36 time and 2^32 bytes of text. This would have a break multiplier of: 
  K = 1/(128*128) = 1/16384
  M = (1 - (1/16384)*(36^2 + 32^2)) / 0.875
    = 0.9810
If this was a known ciphertext attack, and was the first attack on the cipher, the cipher's score would be reduced to under 2% of its pre-attack score.

Clarity and simplicity (Sc)

This is just a number that I give depending on how clearly something is explained, with a very slight influence on how simple the algorithm is. Roughly 90% and 10% respectively. As a rough guide:

Item Score
Reference implementation (C/C++, Pascal, Java, others will be considered). OK, I've decided to be a bit more strict here. A reference implementation should present the following interface: 
  void *sbxStreamAllocState;
  int sbxStreamKey(void *State, u8 *Key, u8 *IV, int Keylen, int IVLen, int Direction);
  int sbxStreamCryptBuf(void *State, u8 *Buffer, int BufLen);
  void sbxStreamFree(void *State);

  void *sbxBlockAllocState;
  int sbxBlockKeyEnc(void *State, u8 *Key, int Keylen, int Rounds, int Direction);
  int sbxBlockCryptBuf(void *State, u8 *Buffer, int BufLen);
  void sbxBlockFree(void *State);
Only implementations presenting this interface (or one that can be easily wrapped by these functions) will receive a speed score. Of course, if it's written in Pascal, Java, etc then an equavalent set of functions should be available for calling by the main sandbox program.

To get a non-zero reference implementation score, an implementation must satisfy the following conditions:
  1. Separation of keying and encryption. Another way of looking at this is that the data to be encrypted can be split up into chunks of arbitrary size, and these chunks are fed (in order) to the encryption function. This must return the same result as feeding the whole lot of data to the function in a single call.
  2. Must be thread-safe. By this I mainly mean that you can have multiple streams of data being encrypted and/or decrypted at the same time. So global variables are out: all the information for the encryption function must be contained in its state data.
Exceptions to #1 above will only be given if the algorithm cannot function under the above requirements (such as CDX-2). There will be no exemptions given for #2. Please email me before you start coding if you have any quaestions about the above. I don't want you spending ages coding when it's not going to be used.

Recommendations for reference implementations:
  • Don't use IVs if it's just combined with the key to produce a unique key. The only times IVs should be used is when they are critical to the algorithm. For example, an RC4 implementation does not "require" an IV, but a block cipher in CBC mode does. There is obviously quite a large grey area for this though :)
  • Try and be cross-platform. Use u8, s16, etc for variables that need to be a specific size, and use the endian-swapping macros END_TOxYY in the sandbox.h file if you do anything endian-specific. This is obviously a bit more work, but will improve the portability a lot.
  • Keep code that's not involved in encryption out of the encryption file. So if you want to have a demo that uses unique features of your particular algorithm, keep it in a different file. The reference implementation should not have a main() function :)
  • If you want to have an optimised version, keep it separate from the reference version. Noone wants to read tweaked C to check your description. However, keeping the interface to the code the same is recommended (ie: make the optimised version a "drop in" replacement for the reference version).
Not following these recommendations won't stop you from having your cipher benchmarked, but will probably result in a lower "reference implementation" score.

Reference implementations that provide no method of encrypting an arbitrary block of data in a thread-safe manner will automatically get zero for the reference implementation part. Implementations that fall into this category include programs where the entire encryption procedure is performed A reference implementation must be capable of both encryption and decryption. And it's got to actually *work* of course. Architecture independance is optional, though encouraged, but it must work on x86-32, with either MSVC or GCC.

You can download the source code for the currently benchmarked ciphers, though no testing has been done outside MSVC.

Currently, I have no way of benchmarking ciphers that do not compile in MSVC, so if you submit such a cipher, it will probably be a reasonable period of time before it gets a score. 		0.25
Full explanation, detailing how encryption and decryption works. Ideally this should only contain psuedocode: any real code should be in the reference implementation. Please don't use any WYSISYG HTML editors or anything like that. Most produce code that only works in IE, and I have to severely damage it to get it to work for everyone else. Plain hand-coded HTML is fine, though, as long as you don't go overboard trying to make it look fancy. 0.30
Fully worked example of encrypting and decrypting a short message. This would link in well with the full explanation. If the cipher is trivial enough (ie: not much more complex than RC4) and well explained, then a worked example is pretty much uunnecessary and these points will be given. I will be the sole judge of whether it is trivial or not :) 0.25
Reasonable amount of plaintext and ciphertext. For a cipher where the algorithm is not supplied, the minimum requirements are:
  • 64 plaintext/ciphertext/key tuples. Each plaintext must be 1024 bytes, and each plaintext/ciphertext must be randomly generated. In addition, there must be at least one plaintext consisting of all zeros, one key consisting of all zeros, and one tuple where the key and plaintext are all zeros.
  • A (random) plaintext encrypted with 64 different keys (producing 64 different ciphertext). Again, the plaintext must be exactly 1024 bytes
The keys used must be the same size as the keys in the challenge.

For a cipher with a provided reference implementation, no text is strictly necessary as I can make up a challenge if you can't be bothered :) If you are submitting a challenge, then the minimum requirement is 4kb of known plaintext encrypted with the same key, and 1kb of challenge ciphertext. However this varies a bit with the type of cipher:
  • For a keystream generator (eg: RC4), the 4kb of plaintext must be the 4kb of plaintext preceeding the 1kb of challenge ciphertext. An obvious choice would be 4kb of zeros. In this case, give the first 4kb of the keystream, then use the next 1kb to encrypt the challenge.
  • For a block cipher (eg: AES), the 4kb of plaintext would be 4kb of random text, encrypted in ECB mode. The challenge would be 1kb of text also encrypted in ECB mode, using the same key. Obviously, this being ECB mode, a check should be done to make sure that none of the "random" blocks in fact match blocks of the challenge text.
  • For a data-dependant stream cipher (eg: a block cipher in CFB mode), the same conditions as a keystream generator apply: 5kb of text (4kb of random garbage + 1kb of challenge text) are encrypted as a whole, and then the first 4kb of plaintext is given.
This is not a definitive list. If a cipher doesn't fit easily into one of these buckets, then just email me and I can probably suggest a good option.

Of course, more plaintext is better, so if you want, feel free to add more varied plaintext in addition to the above.

The challenge plaintext should be a standard English text, though HTML is also acceptable. Any deviation from standard English text must be mentioned. 		0.10
Simplicity. Generally a full score of 0.1 is given here unless the design of the cipher is stupidly complex and unanalysable. 0.10


Overall score

The total overall score after n days is then:
Sp * (1-M*Sb) * Sc * sqrt(n)*ln(1+n)
or just:
Constant * sqrt(n)*ln(1+n)
Simple, eh :)

"Professional" grade

In response to requests to improve the standard of ciphers submitted to the Sandbox, there is an additional level a cipher can get. To be marked as "professional" the submission must be accompanied by a paper that also details a security study of the cipher, for example showing resistance to a differential or linear attack. Additionally, full points (excluding challenge and simplicity) must be obtained for the "Clarity and simplicity" part (though this should be essentially gauranteed if you've taken the time to write up an analysis of the cipher).