Deep learning with DL4J and Domino

by on October 21, 2014

TLDR

We published a public project on Domino that’s pre-configured with deeplearning4j example code, and all setup to compile against the dl4j libraries. You can use it as a template, so you can focus on experimenting with your model, and easily run it on massive machines (32 cores, 240GB of memory) using Domino’s data science platform, without dealing with any infrastructure or setup.


The screenshot above shows several different “Runs” of the dl4j code on Domino, each one executing a different dl4j example. The selected run is the multi-core MNIST example, running on a 4-core machine.

Deep Learning for Java

Deep learning is a popular sub-field of machine learning that has proved effective at learning abstract representations in data sets that are typically only “interprettable” by humans. For example, image processing (face recognition, image search), audio classification, and text analysis.

Deeplearning4j is a “commercial-grade, open-source deep-learning library … meant to be used in business environments, rather than as a research tool.” Since we are building Domino to address the same commercial-grade analytical use cases, we couldn’t wait to learn more about this library.

So we were excited to attend the SF Data Mining Meetup last night, where dl4j creator Adam Gibson spoke about deep learning. One thing he made very clear — over and over — is that deep learning is not a panacea, and it’s not a good solution for all problems. Specifically, his advice was to use it for media (image, video, audio, text) and for time-series analysis (e.g., sensor data feeds). When one person in the audience last night asked about using it for fraud detection, a use case where inspection of the classification logic is critical, Adam literally said something like, “I don’t even want to see you here.” =) We appreciated the honesty and directness.

As great as it was learning more about dl4j, we were even more excited to get dl4j up and running on Domino =)

Although many Domino users work in Python, R, and other scripting languages, at its core, Domino is an arbitrary code executor and as such, it can run any code in basically any language you want to use. So getting a java example up and running was a piece of cake.

Why this is useful

dl4j is powerful and flexible, but not all data scientists and machine learning practioners are software engineers and infrastructure experts. Domino saves you the hassle of infrastructure setup and configuration. For example, you don’t have to worry about getitng an AWS machine set up with all the libraries you need, and getting all your maven dependencies right to compile the dl4j example code.

By using our deeplearning4j-examples project on Domino, all that setup is handled for you, and you can run your code on any hardware you want with one click. At the same time, Domino tracks every run of your code, including your results, so you can reproduce past work and share it with other people. Or you can package your model into a self-service web form so non-technical stakeholders can use it.

How it works

We created a Domino projet with the source code form the deeplearning4j-examples module. Then we made a wrapper shell script, compileAndRun.sh, that compiles the code, and runs java with the main class you specified:

prettyprint lang-bash
#!/bin/bash

mvn package
mkdir tmp
java -cp "target/*:target/dependency/*" -Djava.io.tmpdir=/mnt/tmp $1

You can run it like this: domino run compileAndRun.sh org.deeplearning4j.iris.IrisExample

Your turn to learn deeply…

Feel free to clone the project (domino get nick/deeplearning4j-examples) and copy the structure into your own Domino project.

Edit the source files however you want, or add your own DL4J code. Then run it. Scale up your hardware however you want.

Have fun!

Share