Skip to content

Commit

Permalink
Merge branch 'main' of github.com:memgraph/gqlalchemy into add-regex-…
Browse files Browse the repository at this point in the history
…operator
  • Loading branch information
imilinovic committed Jul 9, 2024
2 parents a5a90d0 + 651240c commit 14fac45
Show file tree
Hide file tree
Showing 11 changed files with 778 additions and 30 deletions.
19 changes: 14 additions & 5 deletions .github/workflows/build-and-test.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
name: Build and Test

env:
MG_VERSION: "2.5.0"
MG_VERSION: "2.17.0"
POETRY_VERSION: "1.2.2"
VM_MAX_MAP_COUNT: "262144"

on:
push:
Expand Down Expand Up @@ -105,7 +106,9 @@ jobs:
path: /var/log/memgraph

build_and_test_windows:
if: github.event.pull_request.draft == false
if: false
# if: github.event.pull_request.draft == false
continue-on-error: true
strategy:
matrix:
python-version: ["3.8", "3.9", "3.10"]
Expand All @@ -123,17 +126,23 @@ jobs:
python -m pip install -U pip
python -m pip install networkx numpy scipy
python -m pip install poethepoet==0.18.1
- uses: Vampire/setup-wsl@v1
- name: Set default WSL version to 2
run: wsl --set-default-version 2
- uses: Vampire/setup-wsl@v3
with:
distribution: Ubuntu-20.04
distribution: Ubuntu-22.04
- name: Update vm.max_map_count
shell: wsl-bash {0} #root shell
run: sysctl -w vm.max_map_count=${{ env.VM_MAX_MAP_COUNT }}
- name: Download, install and run Memgraph under WSL
shell: wsl-bash {0} # root shell
run: |
sudo apt-get update
sudo apt-get -y install python3 python3-pip ipython3
pip3 install networkx numpy scipy
mkdir /memgraph
curl -L https://download.memgraph.com/memgraph/v${{env.MG_VERSION}}/ubuntu-20.04/memgraph_${{env.MG_VERSION}}-1_amd64.deb --output /memgraph/memgraph-community.deb
curl -L https://download.memgraph.com/memgraph/v${{env.MG_VERSION}}/ubuntu-22.04/memgraph_${{env.MG_VERSION}}-1_amd64.deb --output /memgraph/memgraph-community.deb
systemctl mask memgraph
dpkg -i /memgraph/memgraph-community.deb
mkdir /mnt/c/memgraph
runuser -l memgraph -c '/usr/lib/memgraph/memgraph --bolt-port 7687 --bolt-session-inactivity-timeout=300 --data-directory="/mnt/c/memgraph/data" --storage-properties-on-edges=true --storage-snapshot-interval-sec=0 --storage-wal-enabled=false --storage-recover-on-startup=false --storage-snapshot-on-exit=false --telemetry-enabled=false --log-level=TRACE --also-log-to-stderr=true --log-file=/mnt/c/memgraph/memgraph-windows-${{ matrix.python-version }}.log' &
Expand Down
99 changes: 86 additions & 13 deletions docs/how-to-guides/loaders/import-table-data-to-graph-database.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,14 +27,21 @@ data is located, here are two guides on how to import it to Memgraph:

## Loading a CSV file from the local file system

Let's say you have a simple table data in a CSV file stored at
`/home/user/table_data`:
Let's say you have a simple dataset stored in CSV files:

`/home/user/table_data/individual.csv`:
```csv
name,surname,grade
Ivan,Horvat,4
Marko,Andric,5
Luka,Lukic,3
ind_id, name, surname, add_id
1, Ivan, Horvat, 2
2, Marko, Andric, 2
3, Luka, Lukic, 1
```

`/home/user/table_data/address.csv`:
```csv
add_id, street, num, city
1, Ilica, 2, Zagreb
2, Broadway, 12, New York
```

To create a translation from table to graph data, you need to define a **data
Expand Down Expand Up @@ -78,24 +85,37 @@ many_to_many_relations: # intended to be used in case of associative table
column_name:
reference_table:
reference_key:
label:
label: # relationship's label
properties: # list of properties to add to the relationship

```

### One to many

For this example, you don't need all of those fields. You only need to define
`indices` and `one_to_many_relations`. Hence, you have the following YAML file:

```yaml
indices:
example:
- name
address:
- add_id
individual:
- ind_id

name_mappings:
example:
label: PERSON
individual:
label: INDIVIDUAL
address:
label: ADDRESS

one_to_many_relations:
example: []
address: []
individual:
- foreign_key:
column_name: add_id
reference_table: address
reference_key: add_id
label: LIVES_IN
```
In order to read the data configuration from the YAML file, run:
Expand All @@ -114,12 +134,65 @@ make an instance of an `Importer` and call `translate()`.
```python
importer = CSVLocalFileSystemImporter(
data_configuration=parsed_yaml,
path="/home/user/table_data",
path="/home/user/table_data/",
)
importer.translate(drop_database_on_start=True)
```

### Many to many

Relationships can also be defined using a third, associative table.

`/home/user/table_data/tenant.csv`:
```csv
ind_id, add_id, duration
1, 2, 21
2, 2, 3
3, 1, 5
```

We need to extend our data configuration YAML file to include the `many_to_many_relations`, like so:

```
indices:
address:
- add_id
individual:
- ind_id
name_mappings:
individual:
label: INDIVIDUAL
address:
label: ADDRESS
tenant:
column_names_mapping:
duration: years
one_to_many_relations:
address: []
individual: []
many_to_many_relations:
tenant:
foreign_key_from:
column_name: ind_id
reference_table: individual
reference_key: ind_id
foreign_key_to:
column_name: add_id
reference_table: address
reference_key: add_id
label: LIVES_IN
properties:
- duration
```

From here the procedure is the same as before.
In addition to having imported nodes and connected individuals and their addresses, we have also added an edge property.
This property is read from the associative table and is named in accordance with the `name_mappings`.

## Using a cloud storage solution

To connect to Azure Blob, simply change the Importer object you are using. Like
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,6 @@ Class that holds the full description of a single one to many mapping in a table
- `foreign_key` - Foreign key used for mapping.
- `label` - Label which will be applied to the relationship created from this object.
- `from_entity` - Direction of the relationship created from the mapping object.
- `parameters` - Parameters that will be added to the relationship created from this object (Optional).

## ManyToManyMapping Objects

Expand All @@ -44,7 +43,7 @@ Many to many mapping is intended to be used in case of associative tables.
- `foreign_key_from` - Describes the source of the relationship.
- `foreign_key_to` - Describes the destination of the relationship.
- `label` - Label to be applied to the newly created relationship.
- `parameters` - Parameters that will be added to the relationship created from this object (Optional).
- `properties` - List of properties that will be added to the relationship created from this object (Optional).

## TableMapping Objects

Expand Down
Loading

0 comments on commit 14fac45

Please sign in to comment.