anonymize¶

Replaces the MAC addresses in radiotap captures and truncates the frames the way linkside-cap does. For test data from the testlab: own devices only, no real MAC address in the repository, also not in the docs, and the home SSIDs and the EAP realm replaced.

Usage¶

python3 tools/anonymize/anonymize.py in.pcap out.pcap [in2.pcap out2.pcap ...]
python3 tools/anonymize/anonymize.py <session-dir> <output-dir>
python3 tools/anonymize/anonymize.py --map ~/linkside-map.tsv <session-dir> <output-dir>
python3 tools/anonymize/anonymize.py --keep ~/own.txt --ssid homenet=testnet <session-dir> <output-dir>
python3 tools/anonymize/anonymize.py --realm staff.uni.nl=staff.ex.org <session-dir> <output-dir>

• One invocation uses one mapping for all files: capA and capB of a session stay consistent.
• --map writes the mapping as TSV (original, replacement). It contains the real addresses: keep it outside the repo.
• --keep: see Own devices only. --ssid: see SSIDs. --realm: see EAP realms.
• --allow-leftovers: see Check.
• Only pcap (µs and ns, both endians), linktype 127 (radiotap) or 105. pcapng is rejected; convert with editcap -F pcap in.pcapng out.pcap. An empty file, a radio whose capture kept no frame, stays empty.
• Python 3 without dependencies.

Mapping¶

• Each unicast address gets a replacement: the first five bytes from a table of locally administered prefixes (02:00:00:00:01, 02:00:00:00:02, … in order of first appearance), the last byte stays. A prefix that occurs in the input is skipped, so no replaced address equals an original.
• The last byte stays, so the MBSSID derivation (BSSID_i from the transmitted BSSID plus the index in the low bits) and the relation between the BSSIDs of one AP stay correct.
• Addresses that differ only in the first byte get replacements that differ only in the first byte: 02, 06, 0a, … in order of appearance. A UniFi AP derives the BSSIDs of one radio that way (94:…, 9a:…, 9e:…), and the engine groups them on it (sameRadio).
• A control frame's TA with the group bit set is bandwidth signaling (RTS, MU-RTS): the individual address with that bit. It gets the replacement of the individual address, with the bit.
• Broadcast, multicast and 00:00:00:00:00:00 are left unchanged.
• Deterministic: the same input gives the same output and mapping.

Where the addresses are¶

In a session directory the *.pcap files are handled this way. Text files (.json, .status, .txt, .md, .tsv, .csv) get their addresses replaced: aa:bb:… and aa-bb-… always; 12 consecutive hex digits as the JSON value of mac, ap, bssid or a key ending in _mac always, as Mist writes them (the AP's own MAC in mist.json is in no capture); other 12 hex digits only when the address is known from the captures. Case and separator are preserved. session.json loses the recording's label (label, free text the operator typed with Rename…), and is then written again with the app's indentation. Other files and subdirectories are skipped, with a message; hidden files, such as the session's .linkside-session, without one.

Own devices only¶

--keep <file>: one address per line, # starts a comment. A frame whose receiver or transmitter (addr1, addr2; for control frames RA and TA) is a unicast address not in the file is left out, in both passes: its addresses are not mapped either. The other addresses in the header (SA, DA) are mapped as usual. This removes what a context sweep hears from neighbors and the AP's frames to other clients. The file holds real addresses: keep it outside the repo.

SSIDs¶

--ssid old=new, repeatable; both the same length in UTF-8, so no length changes. old=old keeps an SSID.

• Replaced in: SSID elements, also in MBSSID profiles, per-STA profiles and reported frame bodies; the SSID List (84); the OWE transition element; the SSID field of FILS Discovery; text files (exact text, longest first).
• Short SSIDs (CRC-32 of the SSID): in the RNR (TBTT Information of 5, 6 or ≥ 11 bytes), the Short SSID List (255/58) and FILS Discovery. The Short SSID of a replaced SSID becomes that of the replacement. The Short SSID of an SSID that occurs in full in the input stays. Any other Short SSID belongs to a hidden network and becomes the Short SSID of hidden-1, hidden-2, … in order of appearance.

EAP realms¶

--realm old=new, repeatable; both the same length in UTF-8. The realm of an EAP-Response/Identity, the part after @, names the client's institution or provider (on eduroam, staff.<institution>…). Replaced in the identity when it equals old, and as exact text in text files; any other realm stays.

Truncation and FCS¶

• The same rules as linkside-cap (truncation). The wire length stays.
• linkside-cap does not write a frame in which the length block would read outside the bytes; anonymize also leaves it out and counts it as dropped.
• A complete frame with an FCS (radiotap flag) gets a new FCS, if the old one was correct.

Check¶

After replacing, the script searches each output frame for each original address as a 6-byte sequence (also with the group bit set) and as text (12 hex digits, with colons or dashes, either case), the first five bytes of each --keep address in front of a sixth (an own AP's address that is never a transmitter or receiver, such as its base address beside its BSSIDs, in an element the script does not parse), each replaced SSID, the Short SSIDs of the replaced SSIDs and of hidden networks, each replaced realm, and each UUID replaced; and the text files for the addresses, the replaced SSIDs and the replaced realms. A hit means a place the script does not know: it reports frame and offset on stderr and stops with exit status 3. --allow-leftovers lets it continue, for when the hit turns out to be coincidence.

Exit status: 0 good, 1 error in the input, 3 original addresses left.

Limits¶

• Not in the mapping: the UUID-E of WPS (often derived from the MAC address) and device names in WPS and P2P. Test data comes from own devices; check those fields for a new client. SSIDs only with --ssid, realms only with --realm.
• Encrypted parts (protected management frames, encrypted key data, data) stay as they are.
• The name of a Mist AP in session.json, in the reason of a tune to a roam target chosen in New Session ("chosen roam target "), stays: a name Mist gave, which no capture holds. Check it for a recording with a chosen roam target.
• Addresses in vendor elements outside WPS, P2P and OWE, and in a protected frame sent in the clear, are found only when the address occurs elsewhere in the input.