If you’re familiar with Neo4j most likely you’re aware that Neo4j delivers an offical docker image. This short post shows a script allowing you to bundle a Neo4j docker container with the latest version of Bloom – a graph visualization product from Neo4j. Additionally it bundles also the matching version of the APOC library to your docker container.
Neo4j 3.2.0 was released last week at GraphConnect Europe. Among lots of cool new features, unfortunately it has one new “feature” making life of APOC users little bit harder, esp. if you run Neo4j from docker.
Since 3.0 you can enrich Cypher with your own stored procedures. Those are written in Java (or any other JVM language) and get deployed to the /plugins folder. In 3.1 user defined functions were added, followed by aggregate functions in 3.2.
All of them use annotations like @Context GraphDatabaseService db to retrieve a reference to the database itself. For getting deeper into Neo4j’s machinery room one could have @Context GraphDatabaseAPI api allowing you full access. This full access can be abused to break out of the permissions system added in 3.1.
Therefore in 3.2. all procedures and functions run in a sandboxed environment disallowing potentially harmful dependencies like GraphDatabaseAPI from being injected. Using a config option dbms.security.procedures.unrestricted=<procedure names> you can deactivate the sandboxing for a given list of procedures. This config option also allows wildcards like *.
Couple of procedure/functions in APOC depend on using internal components and therefore need to be added as unrestricted procedures. This can be achieved by using apoc.* as value for this config option.
In a regular (aka non-docker) deployment you would just adopt conf/neo4j.conf with that setting.
When starting a Neo4j docker container you can dynamically amend config settings using -e. Everything starting with -e NEO4J_<configKey>=<convifgValue> will be amended to the config file. Be aware that dots need be rewritten with underscore.
There is a gotcha: trying to use `-e NEO4J_dbms_security_procedures_unrestricted=apoc.*` does not work since the shell expands the wildcard * with all file in current directory. Even apoc.\* doesn’t help. I suspect docker internally tries another expansion. What I found finally working is using three backslashes: -e NEO4J_dbms_security_procedures_unrestricted=apoc.\\\*
My typical docker command for firing up a throw-away database to be used for demos and trainings typically looks like this:
docker run --rm -e NEO4J_AUTH=none \
-e NEO4J_dbms_security_procedures_unrestricted=apoc.\\\* \
-v $PWD/plugins:/plugins \
-p 7474:7474 -p 7687:7687 neo4j:3.2.0-enterprise
Of course, the apoc jar file needs to reside in plugins folder of the directory where the command is fired from.
Recently Deutsche Bahn started wifi for everyone travelling with a ICE. So far I had trips where it just worked great, on other trips I could not even connect – not on mobile phone nor on my laptop.
We have great signal strength, wifi on my phone works like a charm. Connecting to the wifi works nicely on the laptop as well. But I cannot connect to the login page for accepting t&c.
I have a Thinkpad X1 Yoga laptop running Ubuntu 16.04. Among a gazillion of other packages docker is installed – mostly for dealing with lots of neo4j databases (of course ;-). The wifi (SSID: WIFIOnICE) itself is not authenticated but upon accessing the first webpage you get redirected to URL http://www.wifionice.de. Here I got a “cannot connect” error message in the browser. DNS lookups however worked fine – on couple of other WIFI issues DNS is a common culprit. Using dig www.wifionice.de I’ve learned that this hostname resolves to IP address 172.18.10.10. Next to check are the routing tables:
stefan@x1 ➜ sudo route -n
Kernel-IP-Routentabelle
Ziel Router Genmask Flags Metric Ref Use Iface
0.0.0.0 172.16.0.1 0.0.0.0 UG 600 0 0 wlp4s0
169.254.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 br-034d1e2af367
172.16.0.0 0.0.0.0 255.255.0.0 U 600 0 0 wlp4s0
172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
172.18.0.0 0.0.0.0 255.255.0.0 U 0 0 0 br-034d1e2af367
Interesting, packages to 172.18.0.0/16 are routed to a weird interface called br-034d1e2af367 and are therefore not set via the wifi device. This finding justifies a loud WTF! This bridge interface is established by docker. Since I’m just a docker user without deep understanding of its internals I cannot really explain its exact purpose. But I don’t have to 😉
It’s good enough to just disable the bridge network interface while doing the wifi authentication:
ihtsudo ifconfig br-034d1e2af367 down
open http://www.wifionice.de in your webbrowser and press the "go online" button
sudo ifconfig br-034d1e2af367 up
After that operation I could use the internet on a ICE train without hassle.
Don’t have this one yet. I seems you can tweak the bridge’s IP number using --bip <CIDR> upon docker startup. But I couldn’t find out the details for now. Happy to read your helpful comments here.
This post is mainly intended as a self-reminder for future train trips. If it’s helpful to others as well I’m more than happy. As a reference I’ve posted this in German language to a question on a forum of Deutsch Bahn as well.
Neo4j has a excellent tool for documenting graph models called graphgists. As the name suggests graphgists are typically stored as github gists in asciidoc format. Additionally to the regular asciidoc you can embed executable cypher and a Neo4j console in a graphgist. For most people it’s perfectly fine hosting their graphgists at github or dropbox. If you want to keep your graphgists private just use a secret gist.
However there are companies with an higher demand for security and privacy that don’t want to expose their stuff onto public networks. Graphgists itself is javascript based and works locally. For the console part it by default connects to http://console.neo4j.org to do the graph operations.
One of my colleagues at Neo Technology recently pointed me to docker.io, a nice LXC based tool to create, maintain, run and share lightweight containers. To get familiar with docker I’ve decided to set up a small project to make graphgists and Neo4j console available in a docker container. This approach allows to handle with your graphgists 100% locally – nothing leaves the container.
Docker containers are assembled by a cookbook called a dockerfile. It specifies the container you want to inherit from and then issue couple of commands to apply your customizations. In my case we need to install a servlet container (tomcat7 here). Neo4j console’s source repo is https://github.com/neo4j-contrib/rabbithole. Based on a recent change it now allows to build a war file ready for deployment into servlet containers. Of course we could have installed maven and clone the repo and exec mvn war:war to build neo4j console. I’ve decided to provide and use a prebuilt war file located at bintray, this removes the need for downloaded a massive number of dependencies for the in-container maven installation. Neo4j console’s war file is deployed as console.war and therefore available in the console context of tomcat.
The graphgist repo is cloned into the location of tomcat’s root context and the location of CONSOLE_URL_BASE is adopted to the locally available neo4j console. Finally tomcat is started as a service. Here’s the full Dockerfile:
[getgit userid=”sarmbruster” repoid=”docker_neo4j_graphgist” path=”Dockerfile” language=”bash”]
Of course you need to install docker locally. The procedure differs among operating system and is documented here. Next is to pull the preconfigured container and run it:
sudo docker pull sarmbruster/neo4j_graphgist
sudo docker run -d -v <absolute_path_for_local_gists>:/var/lib/tomcat7/webapps/ROOT/gists:ro -p 8080:8080 sarmbruster/neo4j_graphgist
This procedure might take some time on first invocation – a good candidate for having a nice espresso.
The second command maps a local directory (it’s crucial to use absolute path) into a in-container directory for accessing the gists. Port 8080 is mapped to the in-container port 8080.
When done your local graphgist setup is finished point your browser to http://localhost:8080. To create a gist, open your favourite text editor and save a <myname>.adoc file in the local gist directory used above when starting docker. For some samples of graphgist files, see https://github.com/neo4j-contrib/graphgist/tree/master/gists.Pointing the browser to http://localhost:8080?myname (without .adoc) should render your graphgist.
Using docker ps and docker stop <containerId> can be used to stop the graphgist containter.
Since I’m absolutely new to docker there might be better and more elegant ways to achieve locally running graphgists. I’m looking forward to read your comments and feedback on this.