use julia language without internet connection (mirror?)

7
votes

Problem: I would like to make julia available for our developers on our corporate network, which has no internet access at all (no proxy), due to sensitive data.

As far as I understand julia is designed to use github. For instance julia> Pkg.init() tries to access: git://github.com/JuliaLang/METADATA.jl

Example: I solved this problem for R by creating a local CRAN repository (rsync) and setting up a local webserver. I also solved this problem for python the same way by creating a local PyPi repository (bandersnatch) + webserver.

Question: Is there a way to create a local repository for metadata and packages for julia?

Thank you in advance. Roman

2
juliamirror

2 Answers

8
votes

Yes, one of the benefits from using the Julia package manager is that you should be able to fork METADATA and host it anywhere you'd like (and keep a branch where you can actually check new packages before allowing your clients to update). You might be one of the first people to actually set up such a system, so expect that you will need to submit some issues (or better yet; pull requests) in order to get everything working smoothly.

See the extra arguments to Pkg.init() where you specify the METADATA repo URL.

If you want a simpler solution to manage I would also think about having a two tier setup where you install packages on one system (connected to the internet), and then copy the resulting ~/.julia directory to the restricted system. If the packages you use have binary dependencies, you might run into problems if you don't have similar systems on both sides, or if some of the dependencies is installed globally, but Pkg.build("Pkgname") might be helpful.

2
votes

This is how I solved it (for now), using second suggestion by ivarne.I use a two tier setup, two networks one connected to internet (office network), one air gapped network (development network).

System information: openSuSE-13.1 (both networks), julia-0.3.5 (both networks)

Tier one (office network)

  • installed julia on an NFS share, /sharename/local/julia.
  • soft linked /sharename/local/bin/julia to /sharename/local/julia/bin/julia
  • appended /sharename/local/bin/ to $PATH using a script in /etc/profile.d/scriptname.sh
  • created /etc/gitconfig on all office network machines: [url "https://"] insteadOf = git:// (to solve proxy server problems with github)
  • now every user on the office network can simply run # julia
  • Pkg.add("PackageName") is then used to install various packages.

The two networks are connected periodically (with certain security measures ssh, firewall, routing) for automated data exchange for a short period of time.

Tier two (development network)

  • installed julia on NFS share equal to tier one.
  • When the networks are connected I use a shell script with rsync -avz --delete to synchronize the .julia directory of tier one to tier two for every user.

Conclusion (so far): It seems to work reasonably well. As ivarne suggested there are problems if a package is installed AND something more than just file copying is done (compiled?) on tier one, the package wont run on tier two. But this can be resolved with Pkg.build("Pkgname").