Unresolved dependency: com.hadoop.gplcompression # hadoop-lzo; 0.4.16 when "sbt update" during firing

Question

After getting the code from git using clone https://github.com/twitter/scalding.git and executing the ./sbt update I get:

 :::::::::::::::::::::::::::::::::::::::::::::: [warn] :: UNRESOLVED DEPENDENCIES :: [warn] :::::::::::::::::::::::::::::::::::::::::::::: [warn] :: com.hadoop.gplcompression#hadoop-lzo;0.4.16: not found 

And later:

 sbt.ResolveException: unresolved dependency: com.hadoop.gplcompression#hadoop-lzo;0.4.16: not found at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:213) at sbt.IvyActions$$anonfun$update$1.apply(IvyActions.scala:122) at sbt.IvyActions$$anonfun$update$1.apply(IvyActions.scala:121) at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:116) 

I deleted the ~/.sbt , installed the latest Scala and the cloned repository again. The problem remains.

My configuration:

 [ dk@localhost scalding]$ scala -version Scala code runner version 2.10.3 -- Copyright 2002-2013, LAMP/EPFL [ dk@localhost scalding]$ uname -a Linux localhost.localdomain 2.6.32-431.3.1.el6.x86_64 #1 SMP Fri Jan 3 21:39:27 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux CentOS 6.5 (Final) 

What could be wrong with ./sbt update ? Should I install hadoop-lzo separately and install CLASSPATH to install Scalding? Any other solutions?

Here is the complete installation log:

 [ dk@localhost wks]$ cd scalding [ dk@localhost scalding]$ ./sbt update Downloading sbt launcher for 0.13.0: From http://typesafe.artifactoryonline.com/typesafe/ivy-releases/org.scala-sbt/sbt-launch/0.13.0/sbt-launch.jar To /home/dk/.sbt/launchers/0.13.0/sbt-launch.jar Getting org.scala-sbt sbt 0.13.0 ... :: retrieving :: org.scala-sbt#boot-app confs: [default] 43 artifacts copied, 0 already retrieved (12440kB/229ms) Getting Scala 2.10.2 (for sbt)... :: retrieving :: org.scala-sbt#boot-scala confs: [default] 5 artifacts copied, 0 already retrieved (24390kB/150ms) [info] Loading project definition from /home/dk/wks/scalding/project [info] Updating {file:/home/dk/wks/scalding/project/}scalding-build... [info] Resolving org.fusesource.jansi#jansi;1.4 ... [info] Done updating. [info] Compiling 3 Scala sources to /home/dk/wks/scalding/project/target/scala-2.10/sbt-0.13/classes... [warn] there were 10 deprecation warning(s); re-run with -deprecation for details [warn] there were 2 feature warning(s); re-run with -feature for details [warn] two warnings found [info] Set current project to scalding (in build file:/home/dk/wks/scalding/) [info] Updating {file:/home/dk/wks/scalding/}scalding-args... [info] Resolving org.fusesource.jansi#jansi;1.4 ... [info] Done updating. [info] Updating {file:/home/dk/wks/scalding/}scalding-date... [info] Resolving org.fusesource.jansi#jansi;1.4 ... [info] Done updating. [info] Updating {file:/home/dk/wks/scalding/}maple... [info] Resolving org.fusesource.jansi#jansi;1.4 ... [info] Done updating. [info] Updating {file:/home/dk/wks/scalding/}scalding... [info] Resolving org.fusesource.jansi#jansi;1.4 ... [info] Done updating. [info] Updating {file:/home/dk/wks/scalding/}scalding-core... [info] Resolving org.fusesource.jansi#jansi;1.4 ... [info] Done updating. [info] Updating {file:/home/dk/wks/scalding/}scalding-json... [info] Resolving org.fusesource.jansi#jansi;1.4 ... [info] Done updating. [info] Updating {file:/home/dk/wks/scalding/}scalding-parquet... [info] Resolving com.hadoop.gplcompression#hadoop-lzo;0.4.16 ... [warn] module not found: com.hadoop.gplcompression#hadoop-lzo;0.4.16 [warn] ==== local: tried [warn] /home/dk/.ivy2/local/com.hadoop.gplcompression/hadoop-lzo/0.4.16/ivys/ivy.xml [warn] ==== public: tried [warn] http://repo1.maven.org/maven2/com/hadoop/gplcompression/hadoop-lzo/0.4.16/hadoop-lzo-0.4.16.pom [warn] ==== snapshots: tried [warn] http://oss.sonatype.org/content/repositories/snapshots/com/hadoop/gplcompression/hadoop-lzo/0.4.16/hadoop-lzo-0.4.16.pom [warn] ==== releases: tried [warn] http://oss.sonatype.org/content/repositories/releases/com/hadoop/gplcompression/hadoop-lzo/0.4.16/hadoop-lzo-0.4.16.pom [warn] ==== Concurrent Maven Repo: tried [warn] http://conjars.org/repo/com/hadoop/gplcompression/hadoop-lzo/0.4.16/hadoop-lzo-0.4.16.pom [warn] ==== Clojars Repository: tried [warn] http://clojars.org/repo/com/hadoop/gplcompression/hadoop-lzo/0.4.16/hadoop-lzo-0.4.16.pom [warn] ==== Twitter Maven: tried [warn] http://maven.twttr.com/com/hadoop/gplcompression/hadoop-lzo/0.4.16/hadoop-lzo-0.4.16.pom [info] Resolving org.fusesource.jansi#jansi;1.4 ... [warn] :::::::::::::::::::::::::::::::::::::::::::::: [warn] :: UNRESOLVED DEPENDENCIES :: [warn] :::::::::::::::::::::::::::::::::::::::::::::: [warn] :: com.hadoop.gplcompression#hadoop-lzo;0.4.16: not found [warn] :::::::::::::::::::::::::::::::::::::::::::::: [info] Updating {file:/home/dk/wks/scalding/}scalding-jdbc... [info] Resolving org.fusesource.jansi#jansi;1.4 ... [info] Done updating. [info] Updating {file:/home/dk/wks/scalding/}scalding-avro... [info] Resolving org.fusesource.jansi#jansi;1.4 ... [info] Done updating. [info] Updating {file:/home/dk/wks/scalding/}scalding-commons... [info] Resolving com.hadoop.gplcompression#hadoop-lzo;0.4.16 ... [warn] module not found: com.hadoop.gplcompression#hadoop-lzo;0.4.16 [warn] ==== local: tried [warn] /home/dk/.ivy2/local/com.hadoop.gplcompression/hadoop-lzo/0.4.16/ivys/ivy.xml [warn] ==== public: tried [warn] http://repo1.maven.org/maven2/com/hadoop/gplcompression/hadoop-lzo/0.4.16/hadoop-lzo-0.4.16.pom [warn] ==== snapshots: tried [warn] http://oss.sonatype.org/content/repositories/snapshots/com/hadoop/gplcompression/hadoop-lzo/0.4.16/hadoop-lzo-0.4.16.pom [warn] ==== releases: tried [warn] http://oss.sonatype.org/content/repositories/releases/com/hadoop/gplcompression/hadoop-lzo/0.4.16/hadoop-lzo-0.4.16.pom [warn] ==== Concurrent Maven Repo: tried [warn] http://conjars.org/repo/com/hadoop/gplcompression/hadoop-lzo/0.4.16/hadoop-lzo-0.4.16.pom [warn] ==== Clojars Repository: tried [warn] http://clojars.org/repo/com/hadoop/gplcompression/hadoop-lzo/0.4.16/hadoop-lzo-0.4.16.pom [warn] ==== Twitter Maven: tried [warn] http://maven.twttr.com/com/hadoop/gplcompression/hadoop-lzo/0.4.16/hadoop-lzo-0.4.16.pom [info] Resolving org.fusesource.jansi#jansi;1.4 ... [warn] :::::::::::::::::::::::::::::::::::::::::::::: [warn] :: UNRESOLVED DEPENDENCIES :: [warn] :::::::::::::::::::::::::::::::::::::::::::::: [warn] :: com.hadoop.gplcompression#hadoop-lzo;0.4.16: not found [warn] :::::::::::::::::::::::::::::::::::::::::::::: [info] Updating {file:/home/dk/wks/scalding/}scalding-repl... [info] Resolving org.codehaus.jackson#jackson-core-asl;1.8.8 ... [info] Done updating. sbt.ResolveException: unresolved dependency: com.hadoop.gplcompression#hadoop-lzo;0.4.16: not found at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:213) at sbt.IvyActions$$anonfun$update$1.apply(IvyActions.scala:122) at sbt.IvyActions$$anonfun$update$1.apply(IvyActions.scala:121) at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:116) at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:116) at sbt.IvySbt$$anonfun$withIvy$1.apply(Ivy.scala:104) at sbt.IvySbt.sbt$IvySbt$$action$1(Ivy.scala:51) at sbt.IvySbt$$anon$3.call(Ivy.scala:60) at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:98) at .... .... many lines of exception stack-trace are skipped ... .... [error] (scalding-commons/*:update) sbt.ResolveException: unresolved dependency: com.hadoop.gplcompression#hadoop-lzo;0.4.16: not found [error] (scalding-parquet/*:update) sbt.ResolveException: unresolved dependency: com.hadoop.gplcompression#hadoop-lzo;0.4.16: not found [error] Total time: 53 s, completed Feb 14, 2014 4:51:51 PM [ dk@localhost scalding]$ ./sbt update^C [ dk@localhost scalding]$ uname -a Linux localhost.localdomain 2.6.32-431.3.1.el6.x86_64 #1 SMP Fri Jan 3 21:39:27 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux [ dk@localhost scalding]$ scala -version Scala code runner version 2.10.3 -- Copyright 

Answer 1

SBT cannot extract com.hadoop.gplcompression#hadoop-lzo;0.4.16 from resolvers defined in build.sbt

Adding maven.twttr.com: "Twitter Maven Repo" at "http://maven.twttr.com" should work.

Answer 2

I don't think this will work anymore, use: " https://maven.twttr.com ". (Pay attention to http s ). They changed it in January 2018.

Hope this helps