Categories
Uncategorized

WiFi at Deutsche Bahn + Ubuntu + Docker == trouble

Recently Deutsche Bahn started wifi for everyone travelling with a ICE. So far I had trips where it just worked great, on other trips I could not even connect – not on mobile phone nor on my laptop.

Today it was different

We have great signal strength, wifi on my phone works like a charm. Connecting to the wifi works nicely on the laptop as well. But I cannot connect to the login page for accepting t&c.

What happened – the analysis

I have a Thinkpad X1 Yoga laptop running Ubuntu 16.04. Among a gazillion of other packages docker is installed – mostly for dealing with lots of neo4j databases (of course ;-). The wifi (SSID: WIFIOnICE) itself is not authenticated but upon accessing the first webpage you get redirected to URL http://www.wifionice.de. Here I got a “cannot connect” error message in the browser. DNS lookups however worked fine – on couple of other WIFI issues DNS is a common culprit. Using dig www.wifionice.de I’ve learned that this hostname resolves to IP address 172.18.10.10. Next to check are the routing tables:

stefan@x1 ➜  sudo route -n
Kernel-IP-Routentabelle
Ziel Router Genmask Flags Metric Ref Use Iface
0.0.0.0 172.16.0.1 0.0.0.0 UG 600 0 0 wlp4s0
169.254.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 br-034d1e2af367
172.16.0.0 0.0.0.0 255.255.0.0 U 600 0 0 wlp4s0
172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
172.18.0.0 0.0.0.0 255.255.0.0 U 0 0 0 br-034d1e2af367

Interesting, packages to 172.18.0.0/16 are routed to a weird interface called br-034d1e2af367 and are therefore not set via the wifi device. This finding justifies a loud WTF! This bridge interface is established by docker. Since I’m just a docker user without deep understanding of its internals I cannot really explain its exact purpose. But I don’t have to 😉

the workaround

It’s good enough to just disable the bridge network interface while doing the wifi authentication:

ihtsudo ifconfig br-034d1e2af367 down
open http://www.wifionice.de in your webbrowser and press the "go online" button
sudo ifconfig br-034d1e2af367 up

After that operation I could use the internet on a ICE train without hassle.

a more elegant solution

Don’t have this one yet. I seems you can tweak the bridge’s IP number using --bip <CIDR> upon docker startup. But I couldn’t find out the details for now. Happy to read your helpful comments here.

finally {}

This post is mainly intended as a self-reminder for future train trips. If it’s helpful to others as well I’m more than happy. As a reference I’ve posted this in German language to a question on a forum of Deutsch Bahn as well.

Categories
Uncategorized

Neo4j cluster and firewalls

This article summarizes the required and optional communication channels for a Neo4j enterprise cluster and provides some sample rules. For the firewall rules I’m using Ubuntu’s ufw package.

As a sample setup we assume the following servers. All of them have 2 network interfaces, eth0 is for communication with outside world, eth1 is for cluster communication.

name eth0 ip address eth1 ip address
server1 172.16.0.1 192.168.1.1
server2 172.16.0.2 192.168.1.2
server3 172.16.0.3 192.168.1.3

access to REST interfaces

By default Neo4j listens on port 7474 for http and on 7473 for https style requests to the rest api. To allow for remote access you need to set in neo4j-server.properties:

org.neo4j.server.webserver.address=172.16.0.1 // or 0.0.0.0 for all interfaces

Inbound access needs to be granted using

ufw allow in on eth0 proto tcp from any to any port 7474

When using SSL the rule needs to hit port 7473 instead.

cluster communication

A Neo4j cluster uses two different communication channels: one for cluster management (joining/leaving the cluster, master elections, etc.) and one for transaction propagation. By default ports 5001 and 6001 are used. On all cluster members we need to allow inbound and outbound traffic for these to the other cluster members:

# cluster management: inbound and outbound
ufw allow in on eth1 proto tcp from 192.168.1.0/24 to any port 5001
ufw allow out on eth1 proto tcp to 192.168.1.0/24 port 5001

# transaction propagation: inbound and outbound
ufw allow in on eth1 proto tcp from 192.168.1.0/24 to any port 6001
ufw allow out on eth1 proto tcp to 192.168.1.0/24 port 6001

online backup

Neo4j enterprise supports online backup – the default port for this 6362. To enable remote backup you need to set in neo4j.properties:

online_backup_server=192.168.1.1:6362 // or 0.0.0.0:6362 for listening also on eth0

The corresponding ufw command is:

ufw allow out on eth1 proto tcp to 192.168.1.0/24 port 6362
ufw allow in on eth1 proto tcp from 192.168.1.0/24 to any port 6362

remote shell

This is pretty tricky. Under the hoods neo4j remote shell uses Java RMI. When a new connection is established the client communicates with server:1337. During this control session a secondary port is negotiated – unfortunately a random port is used. So the client opens a second connection to the server with the negotiated port – therefore we have to open up basically all ports for the ip addresses acting as shell clients:

ufw allow in on eth1 proto tcp from  
ufw allow out on eth1 proto tcp to 

There might be a more sophisticated approach by implementing a custom RMISocketFactory and register it with the JVM as described in the JVM docs. I have not yet tried this, so if you have explored that path yourself I’d appreciate to hear your solution to this.

Categories
Uncategorized

quick tooling tip for hacking Cypher statements – Linux only

When developing Cypher statements for a Neo4j based application there are multiple ways to do this.

A lot of people (including myself) love the new Neo4j browser shipped with 2.0 and subsequent releases. This is a nicely built locally running web application running in your browser. At the top users can easily type their Cypher code and see results after executing, either in tabular form or as a visualization enabling to click through.

Neo4j 2.0 Browser

Another way is to use the command line and either go with neo4j-shell or use the REST interface by a command line client like cURL or more conveniently httpie (which I’ve previously blogged about).

Typically while building a Cypher statement you take a lot of cycles to hack a little bit, test if it runs, hack a little bit, test, …. This cycle can be improved by automating execution as soon as the file containing the cypher statement has hanged.

Linux comes with a kernel feature called inotify that reports file system changes to applications. On Ubuntu/Debian there is a package called inotify-hookable available offering a convenient way to set up tracking for a specific file or directory and take a action triggered by a change in the file/directory.

Assume you want to quickly develop a complex cypher statement in $HOME/myquery.cql. Set up monitoring using:

inotify-hookable -c ~/myquery.cql -c "(~/neo4j-enterprise-2.0.1/bin/neo4j-shell < ~/myquery.cql)"

Using your text editor of choice open $HOME/myquery.cql and change your code. After saving the statement will be automatically executed and you get instantly feedback.

Categories
Uncategorized

running Neo4j graphgists locally with docker.io

Neo4j has a excellent tool for documenting graph models called graphgists. As the name suggests graphgists are typically stored as github gists in asciidoc format. Additionally to the regular asciidoc you can embed executable cypher and a Neo4j console in a graphgist. For most people it’s perfectly fine hosting their graphgists at github or dropbox. If you want to keep your graphgists private just use a secret gist.

However there are companies with an higher demand for security and privacy that don’t want to expose their stuff onto public networks. Graphgists itself is javascript based and works locally. For the console part it by default connects to http://console.neo4j.org to do the graph operations.

One of my colleagues at Neo Technology recently pointed me to docker.io, a nice LXC based tool to create, maintain, run and share lightweight containers. To get familiar with docker I’ve decided to set up a small project to make graphgists and Neo4j console available in a docker container. This approach allows to handle with your graphgists 100% locally – nothing leaves the container.

How the docker container is built up

Docker containers are assembled by a cookbook called a dockerfile. It specifies the container you want to inherit from and then issue couple of commands to apply your customizations. In my case we need to install a servlet container (tomcat7 here). Neo4j console’s source repo is https://github.com/neo4j-contrib/rabbithole. Based on a recent change it now allows to build a war file ready for deployment into servlet containers. Of course we could have installed maven and clone the repo and exec mvn war:war to build neo4j console. I’ve decided to provide and use a prebuilt war file located at bintray, this removes the need for downloaded a massive number of dependencies for the in-container maven installation. Neo4j console’s war file is deployed as console.war and therefore available in the console context of tomcat.

The graphgist repo is cloned into the location of tomcat’s root context and the location of CONSOLE_URL_BASE is adopted to the locally available neo4j console. Finally tomcat is started as a service. Here’s the full Dockerfile:

[getgit userid=”sarmbruster” repoid=”docker_neo4j_graphgist” path=”Dockerfile” language=”bash”]

How to use the docker container

Of course you need to install docker locally. The procedure differs among operating system and is documented here. Next is to pull the preconfigured container and run it:

sudo docker pull sarmbruster/neo4j_graphgist
sudo docker run -d -v <absolute_path_for_local_gists>:/var/lib/tomcat7/webapps/ROOT/gists:ro -p 8080:8080 sarmbruster/neo4j_graphgist

This procedure might take some time on first invocation – a good candidate for having a nice espresso.

The second command maps a local directory (it’s crucial to use absolute path) into a in-container directory for accessing the gists. Port 8080 is mapped to the in-container port 8080.

When done your local graphgist setup is finished point your browser to http://localhost:8080. To create a gist, open your favourite text editor and save a <myname>.adoc file in the local gist directory used above when starting docker. For some samples of graphgist files, see https://github.com/neo4j-contrib/graphgist/tree/master/gists.Pointing the browser to http://localhost:8080?myname (without .adoc) should render your graphgist.

Using docker ps and docker stop <containerId> can be used to stop the graphgist containter.

closing words

Since I’m absolutely new to docker there might be better and more elegant ways to achieve locally running graphgists. I’m looking forward to read your comments and feedback on this.

 

Categories
Uncategorized

Gnome and CTI (computer-telephone-integration)

On my desk, there’s a Siemens Gigaset SX353 connected to the desktop PC via USB. There’s a nice command line tool for managing the telephone, esp. dialing numbers is possible using

gigacontr --dev /dev/ttyGB0 --dial 1 <number> 10

Unfortunately when passing in a international number with the “+” notation, e.g. +49163123456 the phone will not use the “+”. This could be easily solved with a small python wrapper script gigadial.py:

#!/usr/bin/python
import sys, os
assert len(sys.argv)==2
device = "/dev/ttyGB0"
internal_number = "10"
command = "/usr/local/sbin/gigacontr --dev %s --dial 1 %s %s"
number = sys.argv[1]

number = number.replace("+", "00")
if len(number) > 8 and number[0:2]=='49':
	number = "00%s" % (number)

command = command % (device, number, internal_number)
os.system(command)

Gnome supports configuring a handler for callto: URLs. Using gconf-editor modify the setting /desktop/gnome/url-handlers/callto/command must be set to

<path-to>/gigadial.py %s

To use Thunderbird’s addressbook with this, enable in Tool | Additional Settings | Misc the option “insert callto: link for phonenumbers”. When viewing contact data, the phone numbers show up as links. When clicking the phone number, the phone dials that number. Cool!

Even cooler: For Firefox, there the wonderful Telify addon that finds phone number in webpages and converts them to links. Telify must be configured to use callto: URLs instead of the default tel: URLS.