CyberChef is really useful, and it runs locally, so you never have to send your sensitive data to some random server anymore. The “recipes” feature is quite powerful, too, but I think my favorite thing about it is that I can often just paste random data in and run the “Magic” script and it’ll try to guess the data format for me.
> you never have to send your sensitive data to some random server anymore
If you live in the UK you are already sending one entire month of your full personal communications, and three months of your communications metadata, to this government organisation. Pardon the off-topic rant.
No. No meaningful reforms happened. The secret domestic warrantless surveillance infrastructure that exists in 2024 is either far more technically impressive or terrifying than the equivalent in place in 2013, depending on your perspective.
It uses a combination of magic bytes (like the `file` command), entropy analysis and character frequency detection to determine whether an output is likely to be of interest to the user.
The file type mechanism is written here[0]. There's a list of all signatures we detect here[1]. The magic implementation is here[2].
[1] If you click on it they will be able to track you down via your IP address and super seekret cyberspy-fu. Just kidding. .... or am I? Actually I really am. I have no way of knowing either way. Or do I? I mean, how would you know? I really don't though. At least as far as you know.
If you click on it they will be able to track you down via your IP address and super seekret cyberspy-fu. Just kidding. .... or am I? Actually I really am. I have no way of knowing either way. Or do I? I mean, how would you know? I really don't though. At least as far as you know.
GCHQ was the cover name for Bletchley Park but the organisation's name at the time was GC&CS which was established in 1919. So very old and surprisingly predates WW2
I'd have to check my notes from a 40 year old interview with a former Bletchley Park code breaker .. as I recall ...
The official legit name of GC&CS was Government Code and Cypher School .. but IIRC at the time (1919 post WWI) the official name was coined it was putting out letterheads and contracts as G<something> Copper & Cable Services.
That's a dim recollection of what may have been one of many inside jokes | chuckles from WWII Bletchley as retold to an Australia some 30 odd years later so YMMV.
It's in keeping with keeping secrets from the general public & foreign agents via a Boring Name.
The British | Commonwealth WWII company front for their pre Manhatten Project nuclear programme was Tube Alloys .. so they did like a dull metals related cover name.
I've always been amused by these boring sounding cover names. In the USSR, nuclear related work was administered by the "Ministry of Medium Machine Building".
And The Manhattan Project was originally the "Manhattan Engineering District". Although that sounds a little bit exotic, at least to my ears.
I love how you can build complex operations from each of the simple primitives like From Hex->Uppercase->XOR->Remove NULL->To Hex and save them as a recipe. It reminds me of the Linux command line where each program is simple but you can pipe the output of one into another to create a chain that can perform complex data processing.
Literally one of my favorite tools for years. The recipe idea and chaining, plus the crazy list of ingredients is super useful if anybody does CTFs.. much faster than throwing together some bits of Python
I use it for deobfuscation, decryption, base64 en/decode and and sometimes just for converting hex to readable strings or vice versa via copy paste and no effort. It can switch up endianness, sort out network packet data, etc.
For even less effort there is a MAGIC function where you just dump some bytes (sometimes with a from/to hex block first) and it tries to make sense of them for you.
It saves me writing custom C or Python scripts every time I want to manipulate or analyze some data.
I use it for parsing/formatting out text data instead of Excel because I don't have access to a shell and my JavaScript is rusty enough that it is quicker to put regexes in CyberChef than use jsfiddle. Before I found it I was using an online regex tool for that; simple dedupe, and line counting made it worthwhile shifting.
others have commented, but it's really pretty varied.
- literally yesterday used it for building a payload for a vuln that used a java deserialization vulnerability against a known private key for AES encryption. I built the serialized payload using ysoserial as per normal, but then b64 encoded that, then encrypted with the known key and urlencoded the final output.. then I can simply hand that off to curl or burp or whatever.
- sometimes it's as simple as other said, encode/decode chains of various formats -- it's just faster especially if I'm not sure how it was encoded, but I recognize particular patterns (e.g.: url encode, b64 super easy to recognize visually, so plug and play of recipes makes it fast to try stuff out and tune).
I find CyberChef immensely when I'm doing RE work that for things that I'd otherwise have to write little snippets of code to do. I can get a lot of the simple text manipulation stuff done w/ hexdump, cut, tr, sed, etc. I find CyberChef easier to use when I'm doing operations on binary blobs.
I particularly like easily doing encryption and decryption. Lately I seem to find many "secrets" (database connection strings, API keys, etc) in software I'm RE'ing stored as base64-encoded AES-encrypted blobs w/ the key sitting right beside them as a base64-encoded blob.
Immensely useful, I assume you mean - but you missed a word!
Useless commentary aside, could you give some examples of what you’re talking about in your second paragraph? Sounds like the sort of thing I’d enjoy reading!
You might be interested in trying our Easy Data Transform software. It is general purpose data wrangling software, rather than security specific. But it can do base64 and url encoding (in the 'Decode' transform) as well as 65 other transforms (join, dedupe etc) and it can handle multi-million row datasets. Plus in my (biased) opinion, it has a much nicer UI. Not free though.
If you need to wire together a pipeline of transforms (e.g. merge 2 datasets->remove empty rows->dedupe rows->filter rows) then the visual data flow approach is very powerful. But if you are only doing 1 transform (e.g. base64 encoding) then it doesn't have any real advantages.
I also discovered Ciphey. Neat little tool indeed, but I just found out it's being deprecated. It's mentioned in this issue[1] and being replaced with Ares[2]. Neither could decipher this strange encryption[3] I used it on unfortunately.
60 comments
[ 2.7 ms ] story [ 97.9 ms ] threadIf you live in the UK you are already sending one entire month of your full personal communications, and three months of your communications metadata, to this government organisation. Pardon the off-topic rant.
The file type mechanism is written here[0]. There's a list of all signatures we detect here[1]. The magic implementation is here[2].
[0] https://github.com/gchq/CyberChef/blob/master/src/core/lib/F... [1] https://github.com/gchq/CyberChef/blob/master/src/core/lib/F... [2] https://github.com/gchq/CyberChef/blob/master/src/core/lib/M...
https://www.gchq.gov.uk/ <- this is their website.[1]
[1] If you click on it they will be able to track you down via your IP address and super seekret cyberspy-fu. Just kidding. .... or am I? Actually I really am. I have no way of knowing either way. Or do I? I mean, how would you know? I really don't though. At least as far as you know.
Only if they know Visual Basic.
The official legit name of GC&CS was Government Code and Cypher School .. but IIRC at the time (1919 post WWI) the official name was coined it was putting out letterheads and contracts as G<something> Copper & Cable Services.
That's a dim recollection of what may have been one of many inside jokes | chuckles from WWII Bletchley as retold to an Australia some 30 odd years later so YMMV.
It's in keeping with keeping secrets from the general public & foreign agents via a Boring Name.
The British | Commonwealth WWII company front for their pre Manhatten Project nuclear programme was Tube Alloys .. so they did like a dull metals related cover name.
https://en.wikipedia.org/wiki/Tube_Alloys
And The Manhattan Project was originally the "Manhattan Engineering District". Although that sounds a little bit exotic, at least to my ears.
>reminds me of the Linux command line
quite true. upvoted.
but, oddly enough, it also reminds me of the Unix command line. maybe because:
https://en.m.wikipedia.org/wiki/History_of_Unix
>
https://en.m.wikipedia.org/wiki/History_of_Linux
;)
I haven't used it a ton, but I've found the UI to be clunkly and somewhat difficult to figure out compared to using the shell with jq
For even less effort there is a MAGIC function where you just dump some bytes (sometimes with a from/to hex block first) and it tries to make sense of them for you.
It saves me writing custom C or Python scripts every time I want to manipulate or analyze some data.
- literally yesterday used it for building a payload for a vuln that used a java deserialization vulnerability against a known private key for AES encryption. I built the serialized payload using ysoserial as per normal, but then b64 encoded that, then encrypted with the known key and urlencoded the final output.. then I can simply hand that off to curl or burp or whatever.
- sometimes it's as simple as other said, encode/decode chains of various formats -- it's just faster especially if I'm not sure how it was encoded, but I recognize particular patterns (e.g.: url encode, b64 super easy to recognize visually, so plug and play of recipes makes it fast to try stuff out and tune).
I particularly like easily doing encryption and decryption. Lately I seem to find many "secrets" (database connection strings, API keys, etc) in software I'm RE'ing stored as base64-encoded AES-encrypted blobs w/ the key sitting right beside them as a base64-encoded blob.
Useless commentary aside, could you give some examples of what you’re talking about in your second paragraph? Sounds like the sort of thing I’d enjoy reading!
UK GCHQ's CyberChef - https://news.ycombinator.com/item?id=38790631 - Dec 2023 (2 comments)
CyberChef 10 - https://news.ycombinator.com/item?id=35265228 - March 2023 (2 comments)
CyberChef – The Cyber Swiss Army Knife - https://news.ycombinator.com/item?id=32699420 - Sept 2022 (24 comments)
CyberChef – The Cyber Swiss Army Knife - https://news.ycombinator.com/item?id=29982286 - Jan 2022 (54 comments)
CyberChef – Cyber Swiss Army Knife - https://news.ycombinator.com/item?id=20767183 - Aug 2019 (59 comments)
CyberChef - The Cyber Swiss Army Knife - https://news.ycombinator.com/item?id=20543810 - July 2019 (1 comment)
CyberChef – The Cyber Swiss Army Knife - https://news.ycombinator.com/item?id=13099687 - Dec 2016 (1 comment)
CyberChef – A Cyber Swiss Army Knife - https://news.ycombinator.com/item?id=13056254 - Nov 2016 (139 comments)
Maybe it's deemed complete but then I'd wish they just add a message saying so.
If you need to wire together a pipeline of transforms (e.g. merge 2 datasets->remove empty rows->dedupe rows->filter rows) then the visual data flow approach is very powerful. But if you are only doing 1 transform (e.g. base64 encoding) then it doesn't have any real advantages.
protip: Open the JS console (F12 / inspect) and start the CyberChef challenges!
43 6f 6e 67 72 61 74 75 6c 61 74 69 6f 6e 73 2c 20 79 6f 75 20 68 61 76 65 20 63 6f 6d 70 6c 65 74 65 64 20 43 79 62 65 72 43 68 65 66 20 63 68 61 6c 6c 65 6e 67 65 20 23 31 21 0a 0a 54 68 69 73 20 63 68 61 6c 6c 65 6e 67 65 20 65 78 70 6c 6f 72 65 64 20 68 65 78 61 64 65 63 69 6d 61 6c 20 65 6e 63 6f 64 69 6e 67 2e 20 54 6f 20 6c 65 61 72 6e 20 6d 6f 72 65 2c 20 76 69 73 69 74 20 77 69 6b 69 70 65 64 69 61 2e 6f 72 67 2f 77 69 6b 69 2f 48 65 78 61 64 65 63 69 6d 61 6c 2e 0a 0a 54 68 65 20 63 6f 64 65 20 66 6f 72 20 74 68 69 73 20 63 68 61 6c 6c 65 6e 67 65 20 69 73 20 39 64 34 63 62 63 65 66 2d 62 65 35 32 2d 34 37 35 31 2d 61 32 62 32 2d 38 33 33 38 65 36 34 30 39 34 31 36 20 28 6b 65 65 70 20 74 68 69 73 20 70 72 69 76 61 74 65 29 2e 0a 0a 54 68 65 20 6e 65 78 74 20 63 68 61 6c 6c 65 6e 67 65 20 63 61 6e 20 62 65 20 66 6f 75 6e 64 20 61 74 20 68 74 74 70 73 3a 2f 2f 70 61 73 74 65 62 69 6e 2e 63 6f 6d 2f 47 53 6e 54 41 6d 6b 56 2e
This challenge explored hexadecimal encoding. To learn more, visit wikipedia.org/wiki/Hexadecimal.
The code for this challenge is 9d4cbcef-be52-4751-a2b2-8338e6409416 (keep this private).
The next challenge can be found at https://pastebin.com/GSnTAmkV
Its freaking awesome!
https://github.com/ciphey/ciphey
[1] https://github.com/Ciphey/Ciphey/issues/764
[2] https://github.com/bee-san/Ares
[3] "dEFLWWFKQWxRQW16RnkvbTZML0lsdz09" original text is "hacker". But it is unknown how it's being encrypted.