sbt, ivy, offline work, and weirdness

4
votes

I'm trying to work on an sbt project offline (again). Things almost seem to be ok, but there are strange things that I'm baffled by. Here's what I'm noticing:

I've created an empty sbt project and am considering the following dependencies in build.sbt:

name := "sbtSand"

version := "1.0"

scalaVersion := "2.11.7"

libraryDependencies ++= Seq(
    "joda-time" % "joda-time" % "2.9.1",
    "org.apache.spark" %% "spark-core" % "1.5.2"
)

I've built the project while online, and can see all the packages in [userhome]/.ivy2/cache. The project builds fine. I then turn off wifi, sbt clean and attempt to build. The build fails. I comment out the spark dependency (keeping the joda-time one). Still offline, I run sbt compile. The project builds fine. I put the spark dependency back in, and sbt clean. It again fails to build. I get back online. I can build again.

The sbt output for the failed builds are like: https://gist.github.com/ashic/9e5ebc39ff4eb8c41ffb

The key part of it is:

[info] Resolving org.apache.hadoop#hadoop-mapreduce-client-app;2.2.0 ... [warn] Host repo1.maven.org not found. url=https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce-client-app/2.2.0/hadoop-mapreduce-client-app-2.2.0.pom [info] You probably access the destination server through a proxy server that is not well configured.

It's interesting that sbt is managing to use the joda-time from ivy cache, but for the spark-core package (or rather its dependency) it wants to reach out to the internet and fails the build. Could anybody please help me understand this, and what I can do so that I can get this to work while fully offline?

1
apache-sparksbtivy
I've also tried adding scalatest in "test". That worked offline too. - ashic
Are you using sbt's -offline flag? - Seth Tisue
If you mean "set offline := true", then yes - I've tried it. From what I gather, it only applies to snapshots, and I'm not using any. - ashic
@ashic Note that joda-time has a mandatory dependency on joda-convert when used from Scala that is defined as optional in its POM. Hence you must declare that dependency yourself. See repo1.maven.org/maven2/joda-time/joda-time/2.9.1/… and search for mandatory. IIRC not including that dependency can lead to strange behaviour that's tricky to trace down, I don't remember why the POM is as it is, but I blame maven ;-) HTH. P.S. Here's an explanation scala-lang.org/old/node/10575 - jotomo

1 Answers

1
votes

It seems the issue is resolved in 0.13.9. I was using 0.13.8. [The 0.13.9 msi for windows seemed to give me 0.13.8, while the 0.13.9.2 msi installed the right version. Existing projects need updating manually to 0.13.9 in build properties.]