swib18-workshop ✎
From LOD to LOUD: making data usable
Linked Open Usable Data (LOUD) extends Linked Open Data (LOD) by focussing on use cases, being as simple as possible, and providing developer friendly web APIs with JSON-LD. The term was coined by Rob Sanderson. This workshop will introduce you to the basic concepts of LOUD, web APIs, and JSON-LD. You'll learn how to publish and document data as LOUD, and how to use that data in different contexts.
In this workshop, we will:
- Convert RDF data into usable JSON-LD
- Index and query the data with Elasticsearch
- Create a simple web application using the data
- Visualize the data with Kibana
- Document the data with Hypothesis annotations
- Use the data with OpenRefine
Audience: librarians and developers working with linked open data.
Requirements: Laptop with Elasticsearch 6.x, OpenRefine 2.8, a text editor, web browser, and a command line with cURL and jsonld.js via node.js. As an alternative, we'll also provide a fully configured virtual machine to workshop participants.
The LOUD workshop will take place on the premises of the German Institute for Adult Education (Deutsches Institut für Erwachsenenbildung, short: DIE :-). The workshop room is on the ground floor and easy to find. You will have to show your passport at the gate. (This security measures apply because there are other federal authorities located on the premises.) Here is a map showing the five-minute walk from the registration desk at Friedrich-Ebert-Stiftung (green marker) to the gate (red marker):
There are three options:
- Install all tools locally into your own operating system (OS)
- Install VirtualBox and use the virtual machine we provide
- For the command-line exercises (part I and II): Use docker (thanks @EnnoMeijers)
With sample commands for Debian-based Linux systems, follow links for others.
Install git:
apt install git
Install cURL:
apt install curl
Install jq:
apt install jq
Install node (8.x or higher):
curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash -
apt install -y nodejs
Install the hbz jsonld-cli fork:
git clone https://github.com/hbz/jsonld-cli.git
cd jsonld-cli
sudo npm install -g
For details, see the setup instructions.
Install Elasticsearch 6.x:
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
apt install apt-transport-https
echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-6.x.list
sudo apt update && sudo apt install elasticsearch
sudo -i service elasticsearch start
Wait a few seconds for Elasticsearch to start up, then open http://localhost:9200/ to verify Elasticsearch is running.
See http://openrefine.org/download.html
Install Kibana:
sudo apt-get install kibana
sudo -i service kibana start
Wait a few seconds for Kibana to start up, then open http://localhost:5601 to verify Kibana is running.
Clone the repo:
git clone https://github.com/hbz/swib18-workshop.git
Go to the repo:
cd swib18-workshop
Install dependencies:
npm install
Run the server (in js/app.js):
npm start
This will serve the local context at http://localhost:3000/context.json.
As an alternative to the manual setup above, we provide a working environment for VirtualBox. VirtualBox is available for all OSes and allows to run a virtual computer in your own OS. The virtual machine provides every tool mentioned under "Local installation", set up as a Linux box.
See: https://www.VirtualBox.org/wiki/Downloads
Start the box. If it doesn't come up with a GUI you may have to install the "virtualbox-qt" package.
Download the 7z-archived (or zipped) virtual machine from http://labs.lobid.org/download/. The size of the packed file is 2.4 GB (3 GB for the zip), unpacked it's 7.5 GB (so make sure you have got around at least 10 GB free space). To decompress the 7z-archived file you need the 7z archiver.
Installation on Debian-based Linux:
sudo apt install p7zip
Unpack:
7z x swib_2018-Workshop_VBox.7z
Decompressing takes about 5 minutes, depending on your hardware. To set it up in your VirtualBox:
- Menu -> Machine -> Add...
- Select the Swib_2018-Workshop_LOUD.vbox file
When you are finished, a virtual machine should appear in the VirtualBox. Start the machine. A new window should appear with Ubuntu booting until you see the graphical login manager. If you get a "kernel panic" instead, try to enable virtualization in your BIOS or EFI.
The password for the user I is "12345".
Note: the keyboard-layout is preconfigured to German. If you want to change this: click on the blue-white icon in the top left corner under "machine", choose "settings" on the right bottom, select "Keyboard" on the left side. Click on the "Layout" tab, unclick the "Use system defaults", "Add" the keyboard layout you need, then push it to first position by selecting that layout and clicking on the arrows. Close the window, you are done.
Normally it's possible to copy 'n' paste between your "normal" OS (aka "host") and the "guest" (the Ubuntu machine). While it's not a mandatory feature it may be handy. If that's not working and you feel you need it: you need to install the "Guest Additions".
Install docker-compose:
apt install docker-compose
Start the container:
docker-compose up
In a new terminal, run a bash in the container:
docker exec -it swib18workshop_swibws_1 bash
Here you can run the command-line exercises from part I and II.