simplify temporal experiment

Author: qishipengqsp

misc/experiments.md

@@ -1,54 +1,48 @@
 # README
 
-This README elaborates how to reproduce the experiments for choke-points analysis.
+This README elaborates how to run the experiments for choke-points analysis.
 
-Note: These two experiments are conducted in docker.
+## Experiments Environment
 
-Docker image: tugraph/tugraph-compile-centos8:1.3.1
+These two experiments are conducted in docker.
+- Docker image: `tugraph/tugraph-compile-centos8:1.3.1`
+- Hardware: Alibaba Cloud `ecs.r6.8xlarge`, 32xIntel Xeon Platinum 8269CY vCPUs, 256GiB RAM
 
-Hardware: Alibaba Cloud ecs.r6.8xlarge 32xIntel Xeon Platinum 8269CY vCPUs, 256GiB RAM 
+## FinBench Datasets
 
-# Temporal Locality in Storage
+- [SF 0.1](oss://tugraph-web/tugraph/datasets/finbench/v0.2.0/sf0.1/sf0.1.tar.xz) with [md5
+  checksum](oss://tugraph-web/tugraph/datasets/finbench/v0.2.0/sf0.1/sf0.1.tar.xz.md5sum)
+- [SF 1](oss://tugraph-web/tugraph/datasets/finbench/v0.2.0/sf1/sf1.tar.xz) with [md5
+  checksum](oss://tugraph-web/tugraph/datasets/finbench/v0.2.0/sf1/sf1.tar.xz.md5sum)
+- [SF 3](oss://tugraph-web/tugraph/datasets/finbench/v0.2.0/sf3/sf3.tar.xz) with [md5
+  checksum](oss://tugraph-web/tugraph/datasets/finbench/v0.2.0/sf3/sf3.tar.xz.md5sum)
+- [SF 10](oss://tugraph-web/tugraph/datasets/finbench/v0.2.0/sf10/sf10.tar.xz) with [md5
+  checksum](oss://tugraph-web/tugraph/datasets/finbench/v0.2.0/sf10/sf10.tar.xz.md5sum)
+- [SF 30](oss://tugraph-web/tugraph/datasets/finbench/v0.2.0/sf30/sf30.tar.xz) with [md5
+  checksum](oss://tugraph-web/tugraph/datasets/finbench/v0.2.0/sf30/sf30.tar.xz.md5sum)
+- [SF 100](oss://tugraph-web/tugraph/datasets/finbench/v0.2.0/sf100/sf100.tar.xz) with [md5
+  checksum](oss://tugraph-web/tugraph/datasets/finbench/v0.2.0/sf100/sf100.tar.xz.md5sum)
 
-The temporal loacality experiment is executed on TuGraph-DB with FinBench SF100 dataset.
+## Temporal Locality in Storage
 
-## Resources
-
-- Package: https://tugraph-web.oss-cn-beijing.aliyuncs.com/tugraph/datasets/finbench/v0.2.0/experiments/temporal_locality.zip
-    
-- Datasets: https://tugraph-web.oss-cn-beijing.aliyuncs.com/tugraph/datasets/finbench/v0.2.0/sf100/sf100.tar.xz
-
-## Experiment Steps
-
-Execute following steps under current doc's directory, `ldbc_finbench_transaction_impls/misc`.
+The temporal locality experiment is executed on TuGraph-DB with FinBench SF100 dataset. The following steps show how to reproduce the temporal locality experiment. Execute following steps under current doc's directory `ldbc_finbench_transaction_impls/misc`.
 
 ### Download Experiment Resources
 
-#### Download Experiment Package
-Download temporal locality experiment package into current directory.
+Download temporal locality experiment package into current directory and download the FinBench sf100 dataset into experiment package directory.
 
 ```shell
 $ wget https://tugraph-web.oss-cn-beijing.aliyuncs.com/tugraph/datasets/finbench/v0.2.0/experiments/temporal_locality.zip
 $ unzip temporal_locality.zip
-```
-
-#### Download FinBench SF100 Dataset
-Download the FinBench sf100 dataset into experiment package directory
-
-```shell
 $ cd ./temporal_locality
 $ wget https://tugraph-web.oss-cn-beijing.aliyuncs.com/tugraph/datasets/finbench/v0.2.0/sf100/sf100.tar.xz
 $ tar -xf ./sf100.tar.xz
 $ cd ..
 ```
 
-### Run Experiment
-
-The following steps show how to reproduce the temporal locality experiment.
+### Update Source Code in Query Implementation
 
-#### Update Source Code and Compile Benchmark
-
-The temporal locality experiment uses procedure instead of cypher to execute ComplexRead1, this requires a modification in the source code of tugraph's implementation. Please update the ComplexRead1Handler in `ldbc_finbench_transaction_impls/tugraph/src/main/java/org/ldbcouncil/finbench/impls/tugraph/TuGraphTransactionDb.java`(start from line: 100) with the following code.
+The experiment uses procedure to execute `ComplexRead1` instead of Cypher, which requires a modification in the `ComplexRead1Handler` in `ldbc_finbench_transaction_impls/tugraph/src/main/java/org/ldbcouncil/finbench/impls/tugraph/TuGraphTransactionDb.java:100` with the following code.
 
 ```java
 public static class ComplexRead1Handler implements OperationHandler<ComplexRead1, TuGraphDbConnectionState> {
@@ -90,56 +84,39 @@ public static class ComplexRead1Handler implements OperationHandler<ComplexRead1
     }
 ```
 
-Compile the implementation.
+### Compile Benchmark Implementation
+
+compile the implementation.
 
 ```shell
 $ cd ..
 $ mvn clean package
 $ cd ./misc
 ```
 
-#### Start Docker Container
+### Start TuGraph-DB in Docker Container
 
-Docker image `tugraph/tugraph-compile-centos8:1.3.1` is required to run tugraph-db for temporal locality experiment.
+To run temporal locality experiment in docker `tugraph/tugraph-compile-centos8:1.3.1`, install `tugraph-4.0.0-1.el8.x86_64.rpm` first.
 
 ```shell
 $ docker pull tugraph/tugraph-compile-centos8:1.3.1
 $ docker run -it -v ./temporal_locality:/root/temporal_locality -p 7071:7071 -p 9091:9091 --name=finbench_temporal_loaclity tugraph/tugraph-compile-centos8:1.3.1 /bin/bash
-$ exit
-```
-
-#### Install TuGraph-DB
-
-Install tugraph-db with `tugraph-4.0.0-1.el8.x86_64.rpm` inside docker container.
-
-```shell
-$ docker exec -it finbench_temporal_loaclity /bin/bash
 $ cd /root/temporal_locality/
 $ rpm -ivh tugraph-4.0.0-1.el8.x86_64.rpm --nodeps
 ```
 
-use the following command to test if tugraph-db is successfully compile and installed.
-```shell
-$ lgraph_server --version
-```
-
-the output should be like this.
+Use the following command to check if it is successfully installed, whose output is like this. 
 
 ```shell
+$ lgraph_server --version
 TuGraph v4.0.0, compiled from "finbench" branch, commit "335335f" (web commit "8253763").
   CPP compiler version: "GNU" "8.4.0".
   Python version : "3.6.9".
 ```
 
-if installed successfully, exit current container
-
-```shell
-$ exit
-```
-
-#### Import FinBench SF100 Data
+### Preprocess SF100 Dataset
 
-To import sf100 dataset into tugraph-db，the snapshot of sf100 dataset must be converted into compatible data format. This can be done outside of docker container.
+Before importing SF100 dataset，we need to preprocess the snapshot of SF100 dataset in terms of the data format. This can be done outside of docker container.
 
 ```shell
 $ cd ./temporal_locality/sf100
@@ -150,65 +127,73 @@ $ python convert_data.py ./sf100/snapshot.bak ./sf100/snapshot
 $ cd ..
 ```
 
-##### Baseline Mode Data Import
-Import the snapshot of sf100 dataset into tugraph-db(baseline).
+
+### Run Baseline Version
+
+Import SF100 dataset with temporal sorting feature disabled. Then start TuGraph-DB server.
 
 ```shell
 $ docker exec -it finbench_temporal_loaclity /bin/bash
 $ cd /root/temporal_locality/sf100/snapshot
 $ lgraph_import -c /root/temporal_locality/baseline/import.conf --overwrite 1 --dir /root/lgraph_db_sf100 --delimiter "|" --v3 0
-$ exit
+$ cd /root/temporal_locality/
+$ lgraph_server -c /root/temporal_locality/lgraph_sf100.json -d start
+$ exit # exit from the container
 ```
 
-##### Optimized Mode Data Import
-Import the snapshot of sf100 dataset into tugraph-db(optimized).
+Load `ComplexRead1` procedure (baseline version) for baseline experiment. Then link sf100 dataset to `./data` directory, and run benchmark
 
 ```shell
-$ docker exec -it finbench_temporal_loaclity /bin/bash
-$ cd /root/temporal_locality/sf100/snapshot
-$ lgraph_import -c /root/temporal_locality/optimized/import.conf --overwrite 1 --dir /root/lgraph_db_sf100 --delimiter "|" --v3 0
-$ exit
+$ cp ./temporal_locality/load_procedure_temporal.sh ../tugraph/scripts
+$ chmod u+x ../tugraph/scripts/load_procedure_temporal.sh
+$ cp ./temporal_locality/baseline/tcr1.cpp ../tugraph/procedures/cpp
+$ ../tugraph/scripts/load_procedure_temporal.sh
+$ cd ../tugraph/
+$ cd ./data
+$ ln -s ../../misc/temporal_locality/sf100 ./sf100
+$ cd ..
+$ bash run.sh ../misc/temporal_locality/benchmark.properties > temporal_locality.log
+$ cd ../misc
 ```
 
-#### Start TuGraph-DB
+The execution and iteration time in `ComplexRead1` is recorded in `temporal_locality.log`.
+Collect and analyze the elapsed time with `./temporal_locality/log_data_collector.py`
+
+```shell
+$ cd ./temporal_locality
+$ python log_data_collector.py ../../tugraph/temporal_locality.log
+$ cd ..
+```
 
-Start tugraph-db server.
+Stop tugraph-db server.
 
 ```shell
 $ docker exec -it finbench_temporal_loaclity /bin/bash
 $ cd /root/temporal_locality/
-$ lgraph_server -c /root/temporal_locality/lgraph_sf100.json -d start
+$ lgraph_server -d stop
 $ exit
 ```
 
-#### Load Procedure
+### Run Optimized Version
 
-Load ComplexRead1's procedure for temporal locality experiment. First copy `load_procedure_temporal.sh` to `ldbc_finbench_transaction_impls/tugraph/scripts`
+Import SF100 dataset with temporal sorting feature enabled. Then start TuGraph-DB server.
 
 ```shell
-$ cp ./temporal_locality/load_procedure_temporal.sh ../tugraph/scripts
-$ chmod u+x ../tugraph/scripts/load_procedure_temporal.sh
-```
-
-##### Load Baseline Procedure
-
-```shell
-$ cp ./temporal_locality/baseline/tcr1.cpp ../tugraph/procedures/cpp
-$ ../tugraph/scripts/load_procedure_temporal.sh
+$ docker exec -it finbench_temporal_loaclity /bin/bash
+$ cd /root/temporal_locality/sf100/snapshot
+$ lgraph_import -c /root/temporal_locality/optimized/import.conf --overwrite 1 --dir /root/lgraph_db_sf100 --delimiter "|" --v3 0
+$ cd /root/temporal_locality/
+$ lgraph_server -c /root/temporal_locality/lgraph_sf100.json -d start
+$ exit # exit from the container
 ```
 
-##### Load Optimized Procedure
+Load `ComplexRead1` procedure (optimized version) for optimized experiment. Then link sf100 dataset to `./data` directory, then run temporal locality benchmark.
 
 ```shell
+$ cp ./temporal_locality/load_procedure_temporal.sh ../tugraph/scripts
+$ chmod u+x ../tugraph/scripts/load_procedure_temporal.sh
 $ cp ./temporal_locality/optimized/tcr1.cpp ../tugraph/procedures/cpp
 $ ../tugraph/scripts/load_procedure_temporal.sh
-```
-
-#### Run Benchmark
-
-Go to `ldbc_finbench_transaction_impls/tugraph`, link sf100 dataset to `./data` directory, then run temporal locality benchmark
-
-```shell
 $ cd ../tugraph/
 $ cd ./data
 $ ln -s ../../misc/temporal_locality/sf100 ./sf100
@@ -217,18 +202,15 @@ $ bash run.sh ../misc/temporal_locality/benchmark.properties > temporal_locality
 $ cd ../misc
 ```
 
-#### Collect Log Data
-
-The execution time and iteration time in ComplexRead1 is recorded in `temporal_locality.log`, collect and analyze the information with `./temporal_locality/log_data_collector.py`
+The execution and iteration time in `ComplexRead1` is recorded in `temporal_locality.log`.
+Collect and analyze the information with `./temporal_locality/log_data_collector.py`
 
 ```shell
 $ cd ./temporal_locality
 $ python log_data_collector.py ../../tugraph/temporal_locality.log
 $ cd ..
 ```
 
-#### Stop TuGraph-DB
-
 Stop tugraph-db server.
 
 ```shell