Error importing scalding in sbt project

Question

I got this error why importing the scalding sbt in my project build.sbt(ref: How to declare dependency on Scalding in sbt project?). Kindly help me out.

lazy val scaldingCore = ProjectRef(uri("https://github.com/twitter/scalding.git"), "scalding-core")
lazy val myProject = project in file(".") dependsOn scaldingCore

Error:Error while importing SBT project:
...
[warn] ==== public: tried [warn]
https://repo1.maven.org/maven2/com/twitter/scalding-core_2.10/0.16.0-SNAPSHOT/scalding-core_2.10-0.16.0-SNAPSHOT.pom [info] Resolving org.scala-lang#scala-compiler;2.10.4 ... [info] Resolving org.scala-lang#scala-reflect;2.10.4 ... [info] Resolving org.scala-lang#jline;2.10.4 ... [info] Resolving org.fusesource.jansi#jansi;1.4 ... [warn] :::::::::::::::::::::::::::::::::::::::::::::: [warn] ::
UNRESOLVED DEPENDENCIES :: [warn] :::::::::::::::::::::::::::::::::::::::::::::: [warn] :: com.twitter#scalding-core_2.10;0.16.0-SNAPSHOT: not found [warn] :::::::::::::::::::::::::::::::::::::::::::::: [warn] [warn] Note: Unresolved dependencies path: [warn] com.twitter:scalding-core_2.10:0.16.0-SNAPSHOT [warn] +- myproject:myproject_2.10:0.1-SNAPSHOT [trace] Stack trace suppressed: run 'last myProject/:update' for the full output. [trace] Stack trace suppressed: run 'last myProject/:ssExtractDependencies' for the full output. [error] (myProject/:update) sbt.ResolveException: unresolved dependency: com.twitter#scalding-core_2.10;0.16.0-SNAPSHOT: not found [error] (myProject/:ssExtractDependencies) sbt.ResolveException: unresolved dependency: com.twitter#scalding-core_2.10;0.16.0-SNAPSHOT: not found

Well it's an unresolved dependency error. You might be missing a resolver for the Scalding project. What does your build.sbt look like? — airudah
I'd recommend trying an SBT plugin I wrote to get started: github.com/danosipov/sbt-scalding-plugin — Dan Osipov

ches ches · Accepted Answer · 2016-04-18T17:55:08

Scalding publishes jars on Sonatype Maven Central, so you really shouldn't need to mess with the Git ProjectRef. You just need to get your sbt resolvers correct so that it can find the jars. Start with this in your build.sbt:

resolvers ++= Seq(
  Resolver.sonatypeRepo("releases"),
  "Concurrent Maven Repo" at "http://conjars.org/repo"
)

Cascading publishes to the Conjars repository and not Central, so you most likely need that additional resolver as shown.

Try those initially, and if you're still getting unresolved errors, you may need to add additional repos that Scalding uses depending on which artifacts you need to depend on—you probably do not need the entire scalding uber-artifact, you can likely trim down to scalding-core, scalding-commons, scalding-repl, maybe others depending on the needs of your project.

So to be clear, instead of that ProjectRef and dependsOn scaldingCore, add the above resolvers and something like this:

libraryDependencies ++= {
  val scaldingVersion = "0.16.0-RC6"

  Seq(
    "com.twitter" %% "scalding-core"    % scaldingVersion
  , "com.twitter" %% "scalding-commons" % scaldingVersion
  , "com.twitter" %% "scalding-repl"    % scaldingVersion
  )
}

And so on.

Error importing scalding in sbt project

1 Answers