Add support for combined datasets and analysis

This commit is contained in:
Jai Parera
2025-06-11 20:34:09 -07:00
parent 541538fcfe
commit 24fc2ed6f7
11 changed files with 1082 additions and 5 deletions

View File

@@ -2,17 +2,21 @@
Run `pip install -r requirements.txt`
Run `setup.sh`
# Tree Generation
## Download Dataset
Download the *September 22 2016* dataset from: https://iotanalytics.unsw.edu.au/iottraces.html#bib18tmc
Download the *September 22 2016* dataset (or others) from: https://iotanalytics.unsw.edu.au/iottraces.html#bib18tmc
Rename the file as data.pcap
Place these into the `data/tar` folder.
Run `extract_tars.sh` which will extract and place the `.pcap` files at the corresponding location inside `data/pcap`.
## Preprocessing Dataset
Run `ExtractDataset.ipynb`, this will take a few minutes
Run `extract_all_datasets.py` which will extract the data from each file in `data/pcap` and turn it into the corresponding `.csv` file inside `data/processed`. This will take a few minutes per file. Combine the data under `data/csv` using `combine_csv.py`. This will overwrite `data/combined/data.csv` which you can use for the decision tree.
## Training