Skip to content

Commit 7cd072b

Browse files
zoomethzoometh
committed
monly readme.md
1 parent 24c0f06 commit 7cd072b

File tree

1 file changed

+137
-151
lines changed

1 file changed

+137
-151
lines changed

‎README.md‎

Lines changed: 137 additions & 151 deletions
Original file line numberDiff line numberDiff line change
@@ -643,9 +643,11 @@ plot_edtf(edtf_span = "ym", edtf_analyse = "category")
643643

644644
The interactive plotly output is [edtf_plotly_category_ym_threats_types.html](https://eamena-project.github.io/eamenaR/results/edtf_plotly_category_ym_threats_types.html)
645645

646-
# Users
646+
# General statistics
647647

648-
The function [`ref_users()`](https://eamena-project.github.io/eamenaR/doc/ref_users) provides basic statistics on the users of the EAMENA database, for example by plotting the cumulative distribution function of the user first registration:
648+
## Users
649+
650+
The function [`ref_db()`](https://eamena-project.github.io/eamenaR/doc/ref_db) provides basic statistics on the users of the EAMENA database, for example by plotting the cumulative distribution function of the user first registration:
649651

650652
```
651653
d <- hash::hash()
@@ -655,11 +657,11 @@ my_con <- RPostgres::dbConnect(drv = RPostgres::Postgres(),
655657
dbname = 'eamena',
656658
host = 'ec2-54-155-109-226.eu-west-1.compute.amazonaws.com',
657659
port = 5432)
658-
d <- ref_users(db.con = my_con,
659-
d = d,
660-
date.after = "2020-08-01",
661-
plot.g = T,
662-
fig.width = 14)
660+
d <- ref_db(db.con = my_con,
661+
d = d,
662+
date.after = "2020-08-01",
663+
plot.g = T,
664+
fig.width = 14)
663665
```
664666

665667
Here we restrict the plot to dates after 2020-08-01 (option `date.after`). The option `plot.g = T` gives this plot:
@@ -671,7 +673,7 @@ Here we restrict the plot to dates after 2020-08-01 (option `date.after`). The o
671673
The total number of users can also be restricted to an interval (options `date.after` and `date.before`), for example limiting the count to the year 2022:
672674

673675
```
674-
d <- ref_users(db.con = my_con,
676+
d <- ref_db(db.con = my_con,
675677
stat.name = "users_date_joined_2",
676678
d = d,
677679
date.after = "2022-01-01",
@@ -724,149 +726,6 @@ The `dist` shows that the geometries are exactly the same, and that there are sl
724726
## BU
725727

726728
The Bulk upload procedure
727-
728-
### BU append
729-
730-
```mermaid
731-
flowchart LR
732-
A[(EAMENA<br>DB)] --1. GeoJSON<br><b>POINT</b>--> C("geojson_kml()"):::eamenaRfunction;
733-
A --1. GeoJSON<br><b>POINT</b>--> F("geojson_shp()"):::eamenaRfunction;
734-
F --2. SHP--> G((GIS));
735-
G --3. create<br><b>POLYGON</b>--> G;
736-
G --4. SHP--> F;
737-
C --2. KML/KMZ--> B((Google<br>Earth));
738-
B --3. create<br><b>POLYGON</b>--> B;
739-
B --4. KML/KMZ--> C;
740-
C --5. GeoJSON<br><b>POLYGON</b>--> D("geojson_csv()"):::eamenaRfunction;
741-
F --5. GeoJSON<br><b>POLYGON</b>--> D:::eamenaRfunction;
742-
D --6. append<br>new geometries--> A;
743-
classDef eamenaRfunction fill:#e7deca;
744-
```
745-
746-
#### Integrating Google Earth geometries
747-
748-
Most of the geometries in EAMENA are POINTS (`Geometry Type` = `Center Point`). The objective is to acquire new geometries, like POLYGONs, created in third part app, like Google Earth or a GIS, and to append them to already existing records in EAMENA.
749-
750-
751-
```mermaid
752-
flowchart LR
753-
A[(EAMENA<br>DB)] --1. GeoJSON<br><b>POINT</b>--> C("geojson_kml()"):::eamenaRfunction;
754-
C --2. KML/KMZ--> B((Google<br>Earth));
755-
B --3. create<br><b>POLYGON</b>--> B;
756-
B --4. KML/KMZ--> C;
757-
C --5. GeoJSON<br><b>POLYGON</b>--> D("geojson_csv()"):::eamenaRfunction;
758-
D --6. append<br>new geometries--> A;
759-
classDef eamenaRfunction fill:#e7deca;
760-
```
761-
762-
<p align="center"> workflow to work with Google Earth </p>
763-
764-
```mermaid
765-
flowchart LR
766-
A[(EAMENA<br>DB)] --1. GeoJSON<br><b>POINT</b>--> E("geojson_shp()"):::eamenaRfunction;
767-
E --2. SHP--> F((GIS));
768-
F --3. create<br><b>POLYGON</b>--> F;
769-
F --4. SHP--> E;
770-
E --5. GeoJSON<br><b>POLYGON</b>--> D("geojson_csv()"):::eamenaRfunction;
771-
D --6. append<br>new geometries--> A;
772-
classDef eamenaRfunction fill:#e7deca;
773-
```
774-
775-
<p align="center"> workflow to work with a GIS </p>
776-
777-
778-
functions:
779-
- [`geojson_kml()`](https://eamena-project.github.io/eamenaR/doc/geojson_kml)
780-
- [`geojson_shp()`](https://eamena-project.github.io/eamenaR/doc/geojson_shp)
781-
- [`geojson_csv()`](https://eamena-project.github.io/eamenaR/doc/geojson_csv)
782-
783-
For example:
784-
785-
1. Export a GeoJSON file from EAMENA (see: [GeoJSON files](https://github.com/eamena-project/eamenaR#geojson-files)), for example **caravanserail.geojson** ([rendered](https://github.com/eamena-project/eamena-arches-dev/blob/main/data/geojson/caravanserail.geojson) | [raw](https://raw.githubusercontent.com/eamena-project/eamena-arches-dev/main/data/geojson/caravanserail.geojson)) Heritage Places.
786-
787-
<a name="geojson_kml"></a>
788-
789-
2. Convert **caravanserail.geojson** to a KML file named 'caravanserail_outKML.kml' with the [`geojson_kml()`](https://eamena-project.github.io/eamenaR/doc/geojson_kml) function, filtering on POINTS[^3]:
790-
791-
```
792-
library(dplyr)
793-
geojson_kml(geom.types = c("POINT"),
794-
geojson.name = "caravanserail_outKML")
795-
```
796-
797-
![](results/geojson_kml_toKML.png)
798-
799-
<a name="bulk_append_3"></a>
800-
801-
3. Open 'caravanserail_outKML' in Google Earth and draw POLYGONS. Name the newly created POLYGONS with the ResourceID of a given HP.
802-
803-
![](results/geojson_kml_toKML_polygon.png)
804-
805-
4. Export as KML ('caravanserail_outKML2.kml')
806-
5. Convert 'caravanserail_outKML2.kml' into GeoJSON with the [`geojson_kml()`](https://eamena-project.github.io/eamenaR/doc/geojson_kml) function selecting only the POLYGONs (ie, the new geometries).
807-
808-
```
809-
geojson_kml(geom.path = geom.path = paste0(system.file(package = "eamenaR"),
810-
"/extdata/caravanserail_outKML2.kml")
811-
geom.types = c("POLYGON"),
812-
geojson.name = "caravanserail_outGeoJSON")
813-
```
814-
815-
The result is new POLYGON geometries (eg. [caravanserail_outGeoJSON.geojson](https://raw.githubusercontent.com/eamena-project/eamenaR/main/results/caravanserail_outGeoJSON.geojson))
816-
817-
818-
6. Convert the GeoJSON POLYGONs geometries to a format compliant with the EAMENA DB, using the [`geojson_csv()`](https://eamena-project.github.io/eamenaR/doc/geojson_csv) function
819-
820-
```
821-
geojson_csv(geom.path = paste0(system.file(package = "eamenaR"),
822-
"/extdata/caravanserail_outGeoJSON.geojson"),
823-
csv.name = "caravanserail_outCSV")
824-
```
825-
826-
The result is a CSV file, [caravanserail_outCSV.csv](https://github.com/eamena-project/eamenaR/blob/main/inst/extdata/caravanserail_outCSV.csv), with the ResourceID and the geometry of each HP. The fields "Location Certainty" and "Geometry Extent Certainty" are filled with default values.
827-
828-
<a name="bulk_append_6"></a>
829-
```
830-
"resourceid","Geometric Place Expression","Location Certainty","Geometry Extent Certainty"
831-
"8db560d5-d17d-40ff-8046-0157b1b698ab","MULTIPOLYGON (((61.4023 30.77373, 61.4019 30.77371, 61.40194 30.77344, 61.40235 30.77345, 61.4023 30.77373)))","High","High"
832-
"b8305141-789e-4aaa-976a-c85859e0870f","MULTIPOLYGON (((51.47507 33.09169, 51.47463 33.09125, 51.47519 33.09086, 51.47561 33.09133, 51.47507 33.09169)))","High","High"
833-
```
834-
835-
7. These new geometries will be uploaded into the EAMENA DB and append to existing HP. But it should be safe to first check that every ResourceID exist in the DB (maybe a newly created POLYGON has a typo in its name). Use the [`uuid_id()`](https://eamena-project.github.io/eamenaR/doc/uuid_id) function, in a loop to confirm the existence of the ResourceID
836-
837-
```
838-
mycsv <- "https://raw.githubusercontent.com/eamena-project/eamenaR/main/inst/extdata/caravanserail_outCSV.csv"
839-
df <- read.csv(mycsv)
840-
for(i in seq(1, nrow(df))){
841-
eamenaid <- df[i, "ResourceID"]
842-
d <- uuid_id(db.con = my_con,
843-
d = d,
844-
id = eamenaid,
845-
disconn = FALSE)
846-
print(paste0(as.character(i), ") ", eamenaid, " <-> ", d$eamenaid))
847-
}
848-
DBI::dbDisconnect(my_con)
849-
```
850-
851-
Will give:
852-
853-
```
854-
[1] "1) 8db560d5-d17d-40ff-8046-0157b1b698ab <-> EAMENA-0192281"
855-
[1] "2) b8305141-789e-4aaa-976a-c85859e0870f <-> EAMENA-0182054"
856-
```
857-
858-
As there are no `NA` in front of the ResourceID, the HP listed in the CSV file exist in the DB.
859-
860-
8. To append these geometries to the DB, use the `-ow append` option in the `import_business_data` function (see the [Arches documentation](https://arches.readthedocs.io/en/5.1/command-line-reference/#import-business-data))
861-
862-
<a name="bu_append_8"></a>
863-
```
864-
python manage.py packages -o import_business_data -s "./data/test/caravanserail_outCSV2.csv" -c "./data/test/Heritage Place.mapping" -ow append
865-
```
866-
867-
Now, each of these two HP has two different kind of geometries: POINT and POLYGON. See for example the whole dataset of caravanserails
868-
[caravanserail_polygon.geojson](https://github.com/eamena-project/eamenaR/blob/main/inst/extdata/caravanserail_polygon.geojson), one of the record rendered ([EAMENA-0192281.geojson](https://github.com/eamena-project/eamenaR/blob/main/inst/extdata/EAMENA-0192281.geojson)) or this latter record in the [EAMENA DB](https://database.eamena.org/en/report/8db560d5-d17d-40ff-8046-0157b1b698ab)[^4].
869-
870729
### BU mapping
871730

872731
Get a BU file (*target*, see ["what is a BU?"](https://github.com/eamena-project/eamena-arches-dev/tree/main/data/bulk#bulk-upload-bu--)) from an already structured file (*source*) with the [list_mapping_bu()](https://eamena-project.github.io/eamenaR/doc/list_mapping_bu) function. This function uses a mapping file to create the equivalences between the source file and the target file
@@ -1027,6 +886,133 @@ The data from this new worksheet can be copied/pasted into a [BU template](https
1027886
</p>
1028887

1029888

889+
### BU append
890+
891+
#### Integrating Google Earth geometries
892+
893+
Most of the geometries in EAMENA are POINTS (`Geometry Type` = `Center Point`). The objective is to acquire new geometries, like POLYGONs, created in third part app, like Google Earth or a GIS, and to append them to already existing records in EAMENA.
894+
895+
896+
```mermaid
897+
flowchart LR
898+
A[(EAMENA<br>DB)] --1. GeoJSON<br><b>POINT</b>--> C("geojson_kml()"):::eamenaRfunction;
899+
C --2. KML/KMZ--> B((Google<br>Earth));
900+
B --3. create<br><b>POLYGON</b>--> B;
901+
B --4. KML/KMZ--> C;
902+
C --5. GeoJSON<br><b>POLYGON</b>--> D("geojson_csv()"):::eamenaRfunction;
903+
D --6. append<br>new geometries--> A;
904+
classDef eamenaRfunction fill:#e7deca;
905+
```
906+
907+
<p align="center"> workflow to work with Google Earth </p>
908+
909+
```mermaid
910+
flowchart LR
911+
A[(EAMENA<br>DB)] --1. GeoJSON<br><b>POINT</b>--> E("geojson_shp()"):::eamenaRfunction;
912+
E --2. SHP--> F((GIS));
913+
F --3. create<br><b>POLYGON</b>--> F;
914+
F --4. SHP--> E;
915+
E --5. GeoJSON<br><b>POLYGON</b>--> D("geojson_csv()"):::eamenaRfunction;
916+
D --6. append<br>new geometries--> A;
917+
classDef eamenaRfunction fill:#e7deca;
918+
```
919+
920+
<p align="center"> workflow to work with a GIS </p>
921+
922+
923+
functions:
924+
- [`geojson_kml()`](https://eamena-project.github.io/eamenaR/doc/geojson_kml)
925+
- [`geojson_shp()`](https://eamena-project.github.io/eamenaR/doc/geojson_shp)
926+
- [`geojson_csv()`](https://eamena-project.github.io/eamenaR/doc/geojson_csv)
927+
928+
For example:
929+
930+
1. Export a GeoJSON file from EAMENA (see: [GeoJSON files](https://github.com/eamena-project/eamenaR#geojson-files)), for example **caravanserail.geojson** ([rendered](https://github.com/eamena-project/eamena-arches-dev/blob/main/data/geojson/caravanserail.geojson) | [raw](https://raw.githubusercontent.com/eamena-project/eamena-arches-dev/main/data/geojson/caravanserail.geojson)) Heritage Places.
931+
932+
<a name="geojson_kml"></a>
933+
934+
2. Convert **caravanserail.geojson** to a KML file named 'caravanserail_outKML.kml' with the [`geojson_kml()`](https://eamena-project.github.io/eamenaR/doc/geojson_kml) function, filtering on POINTS[^3]:
935+
936+
```
937+
library(dplyr)
938+
geojson_kml(geom.types = c("POINT"),
939+
geojson.name = "caravanserail_outKML")
940+
```
941+
942+
![](results/geojson_kml_toKML.png)
943+
944+
<a name="bulk_append_3"></a>
945+
946+
3. Open 'caravanserail_outKML' in Google Earth and draw POLYGONS. Name the newly created POLYGONS with the ResourceID of a given HP.
947+
948+
![](results/geojson_kml_toKML_polygon.png)
949+
950+
4. Export as KML ('caravanserail_outKML2.kml')
951+
5. Convert 'caravanserail_outKML2.kml' into GeoJSON with the [`geojson_kml()`](https://eamena-project.github.io/eamenaR/doc/geojson_kml) function selecting only the POLYGONs (ie, the new geometries).
952+
953+
```
954+
geojson_kml(geom.path = geom.path = paste0(system.file(package = "eamenaR"),
955+
"/extdata/caravanserail_outKML2.kml")
956+
geom.types = c("POLYGON"),
957+
geojson.name = "caravanserail_outGeoJSON")
958+
```
959+
960+
The result is new POLYGON geometries (eg. [caravanserail_outGeoJSON.geojson](https://raw.githubusercontent.com/eamena-project/eamenaR/main/results/caravanserail_outGeoJSON.geojson))
961+
962+
963+
6. Convert the GeoJSON POLYGONs geometries to a format compliant with the EAMENA DB, using the [`geojson_csv()`](https://eamena-project.github.io/eamenaR/doc/geojson_csv) function
964+
965+
```
966+
geojson_csv(geom.path = paste0(system.file(package = "eamenaR"),
967+
"/extdata/caravanserail_outGeoJSON.geojson"),
968+
csv.name = "caravanserail_outCSV")
969+
```
970+
971+
The result is a CSV file, [caravanserail_outCSV.csv](https://github.com/eamena-project/eamenaR/blob/main/inst/extdata/caravanserail_outCSV.csv), with the ResourceID and the geometry of each HP. The fields "Location Certainty" and "Geometry Extent Certainty" are filled with default values.
972+
973+
<a name="bulk_append_6"></a>
974+
```
975+
"resourceid","Geometric Place Expression","Location Certainty","Geometry Extent Certainty"
976+
"8db560d5-d17d-40ff-8046-0157b1b698ab","MULTIPOLYGON (((61.4023 30.77373, 61.4019 30.77371, 61.40194 30.77344, 61.40235 30.77345, 61.4023 30.77373)))","High","High"
977+
"b8305141-789e-4aaa-976a-c85859e0870f","MULTIPOLYGON (((51.47507 33.09169, 51.47463 33.09125, 51.47519 33.09086, 51.47561 33.09133, 51.47507 33.09169)))","High","High"
978+
```
979+
980+
7. These new geometries will be uploaded into the EAMENA DB and append to existing HP. But it should be safe to first check that every ResourceID exist in the DB (maybe a newly created POLYGON has a typo in its name). Use the [`uuid_id()`](https://eamena-project.github.io/eamenaR/doc/uuid_id) function, in a loop to confirm the existence of the ResourceID
981+
982+
```
983+
mycsv <- "https://raw.githubusercontent.com/eamena-project/eamenaR/main/inst/extdata/caravanserail_outCSV.csv"
984+
df <- read.csv(mycsv)
985+
for(i in seq(1, nrow(df))){
986+
eamenaid <- df[i, "ResourceID"]
987+
d <- uuid_id(db.con = my_con,
988+
d = d,
989+
id = eamenaid,
990+
disconn = FALSE)
991+
print(paste0(as.character(i), ") ", eamenaid, " <-> ", d$eamenaid))
992+
}
993+
DBI::dbDisconnect(my_con)
994+
```
995+
996+
Will give:
997+
998+
```
999+
[1] "1) 8db560d5-d17d-40ff-8046-0157b1b698ab <-> EAMENA-0192281"
1000+
[1] "2) b8305141-789e-4aaa-976a-c85859e0870f <-> EAMENA-0182054"
1001+
```
1002+
1003+
As there are no `NA` in front of the ResourceID, the HP listed in the CSV file exist in the DB.
1004+
1005+
8. To append these geometries to the DB, use the `-ow append` option in the `import_business_data` function (see the [Arches documentation](https://arches.readthedocs.io/en/5.1/command-line-reference/#import-business-data))
1006+
1007+
<a name="bu_append_8"></a>
1008+
```
1009+
python manage.py packages -o import_business_data -s "./data/test/caravanserail_outCSV2.csv" -c "./data/test/Heritage Place.mapping" -ow append
1010+
```
1011+
1012+
Now, each of these two HP has two different kind of geometries: POINT and POLYGON. See for example the whole dataset of caravanserails
1013+
[caravanserail_polygon.geojson](https://github.com/eamena-project/eamenaR/blob/main/inst/extdata/caravanserail_polygon.geojson), one of the record rendered ([EAMENA-0192281.geojson](https://github.com/eamena-project/eamenaR/blob/main/inst/extdata/EAMENA-0192281.geojson)) or this latter record in the [EAMENA DB](https://database.eamena.org/en/report/8db560d5-d17d-40ff-8046-0157b1b698ab)[^4].
1014+
1015+
10301016
# Others
10311017

10321018
## Share a GeoJSON geometry

0 commit comments

Comments
 (0)