InvalidPathException on Assembly 2.1.1

[GEverding]

I'm getting this exception after upgrading from 1.2.0 to 2.1.1. Is the λ? How can i handle this?

[error] java.nio.file.InvalidPathException: Malformed input or input contains unmappable characters: shapeless/$tilde$qmark$greater$λ.class
--
246 | [error]     at java.base/sun.nio.fs.UnixPath.encode(UnixPath.java:121)
247 | [error]     at java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:68)
248 | [error]     at java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
249 | [error]     at java.base/java.nio.file.Path.of(Path.java:147)
250 | [error]     at java.base/java.nio.file.Paths.get(Paths.java:69)
251 | [error]     at sbtassembly.Assembly$.$anonfun$reportConflictsMissedByTheMerge$1(Assembly.scala:652)
252 | [error]     at sbtassembly.Assembly$.$anonfun$reportConflictsMissedByTheMerge$4(Assembly.scala:661)
253 | [error]     at scala.collection.TraversableLike.$anonfun$flatMap$1(TraversableLike.scala:293)
254 | [error]     at scala.collection.Iterator.foreach(Iterator.scala:943)
255 | [error]     at scala.collection.Iterator.foreach$(Iterator.scala:943)
256 | [error]     at scala.collection.AbstractIterator.foreach(Iterator.scala:1431)
257 | [error]     at scala.collection.IterableLike.foreach(IterableLike.scala:74)
258 | [error]     at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
259 | [error]     at scala.collection.AbstractIterable.foreach(Iterable.scala:56)
260 | [error]     at scala.collection.TraversableLike.flatMap(TraversableLike.scala:293)
261 | [error]     at scala.collection.TraversableLike.flatMap$(TraversableLike.scala:290)
262 | [error]     at scala.collection.AbstractTraversable.flatMap(Traversable.scala:108)
263 | [error]     at sbtassembly.Assembly$.$anonfun$reportConflictsMissedByTheMerge$3(Assembly.scala:660)
264 | [error]     at scala.collection.parallel.AugmentedIterableIterator.flatmap2combiner(RemainsIterator.scala:133)
265 | [error]     at scala.collection.parallel.AugmentedIterableIterator.flatmap2combiner$(RemainsIterator.scala:130)
266 | [error]     at scala.collection.parallel.immutable.ParVector$ParVectorIterator.flatmap2combiner(ParVector.scala:66)
267 | [error]     at scala.collection.parallel.ParIterableLike$FlatMap.leaf(ParIterableLike.scala:1082)
268 | [error]     at scala.collection.parallel.Task.$anonfun$tryLeaf$1(Tasks.scala:53)
269 | [error]     at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
270 | [error]     at scala.util.control.Breaks$anon$1.catchBreak(Breaks.scala:67)
271 | [error]     at scala.collection.parallel.Task.tryLeaf(Tasks.scala:56)
272 | [error]     at scala.collection.parallel.Task.tryLeaf$(Tasks.scala:50)
273 | [error]     at scala.collection.parallel.ParIterableLike$FlatMap.tryLeaf(ParIterableLike.scala:1078)
274 | [error]     at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.internal(Tasks.scala:170)
275 | [error]     at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.internal$(Tasks.scala:157)
276 | [error]     at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.internal(Tasks.scala:440)
277 | [error]     at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.compute(Tasks.scala:150)
278 | [error]     at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.compute$(Tasks.scala:149)
279 | [error]     at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:440)
280 | [error]     at java.base/java.util.concurrent.RecursiveAction.exec(RecursiveAction.java:194)
281 | [error]     at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
282 | [error]     at java.base/java.util.concurrent.ForkJoinTask.awaitDone(ForkJoinTask.java:436)
283 | [error]     at java.base/java.util.concurrent.ForkJoinTask.join(ForkJoinTask.java:670)
284 | [error]     at scala.collection.parallel.ForkJoinTasks$WrappedTask.sync(Tasks.scala:379)
285 | [error]     at scala.collection.parallel.ForkJoinTasks$WrappedTask.sync$(Tasks.scala:379)
286 | [error]     at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.sync(Tasks.scala:440)
287 | [error]     at scala.collection.parallel.ForkJoinTasks.executeAndWaitResult(Tasks.scala:423)
288 | [error]     at scala.collection.parallel.ForkJoinTasks.executeAndWaitResult$(Tasks.scala:416)
289 | [error]     at scala.collection.parallel.ForkJoinTaskSupport.executeAndWaitResult(TaskSupport.scala:60)
290 | [error]     at scala.collection.parallel.ExecutionContextTasks.executeAndWaitResult(Tasks.scala:555)
291 | [error]     at scala.collection.parallel.ExecutionContextTasks.executeAndWaitResult$(Tasks.scala:555)
292 | [error]     at scala.collection.parallel.ExecutionContextTaskSupport.executeAndWaitResult(TaskSupport.scala:84)
293 | [error]     at scala.collection.parallel.ParIterableLike$ResultMapping.leaf(ParIterableLike.scala:968)
294 | [error]     at scala.collection.parallel.Task.$anonfun$tryLeaf$1(Tasks.scala:53)
295 | [error]     at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
296 | [error]     at scala.util.control.Breaks$anon$1.catchBreak(Breaks.scala:67)
297 | [error]     at scala.collection.parallel.Task.tryLeaf(Tasks.scala:56)
298 | [error]     at scala.collection.parallel.Task.tryLeaf$(Tasks.scala:50)
299 | [error]     at scala.collection.parallel.ParIterableLike$ResultMapping.tryLeaf(ParIterableLike.scala:963)
300 | [error]     at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.compute(Tasks.scala:153)
301 | [error]     at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.compute$(Tasks.scala:149)
302 | [error]     at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:440)
303 | [error]     at java.base/java.util.concurrent.RecursiveAction.exec(RecursiveAction.java:194)
304 | [error]     at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
305 | [error]     at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
306 | [error]     at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
307 | [error]     at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
308 | [error]     at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
309 | [error] (assembly) java.nio.file.InvalidPathException: Malformed input or input contains unmappable characters: shapeless/$tilde$qmark$greater$λ.class

[cpaika]

+1 Also getting this on the lambda character in shapeless on 2.11

[ryan-richt]

Same problem here

[jckim21]

Has anyone found a fix or a workaround for this?

[fnqista]

Hey guys I would like to help, but It is hard to reproduce. Can someone provide the ff info?:

Java version (i.e. 1.8.212)
Scala version (i.e. 2.11.12)
If you know/can get the "sun.jnu.encoding" (via Console println System.getProperty("sun.jnu.encoding"))

Also, obviously, the problem occurs in Linux based the stacktrace of the OP, but if it also happens in Windows, do mention.

[eed3si9n]

Yea. If someone could reproduce it on a public GitHub repo and/or GitHub Actions that would be helpful.

[jpassaro]

Unable to share a detailed repro just yet, but perhaps this will help clarify things?

I encountered this problem in a docker image (private to my company as far as I can tell) in which sun.jnu.encoding=ANSI_X3.4-1968. In my local machine where this doesn't occur, I have the same setting as UTF-8.

Adding -Dsun.jnu.encoding=UTF-8 to my .sbtopts did not fix it.

if more info is still needed to track this down I'll try and provide an example repo and a docker run that reproduces the problem.

[jpassaro]

By the way I can also report that this reproduced with sbt-assembly 2.0.0.

[ssilb4]

same error on shapeless in 2.1.0
1.2.0 is works

[grcanosa-alstom]

I also have the same problem. Any news on how to solve this?

[jpassaro]

maintainers have indicated a need for a reproducible example. if you have one please share it.

(my team has the creation of a mcve for this as a backlog item, but no guarantees on when we'll be able to produce one.)

EDIT: if you need a workaround, try downgrading to 1.2.0

[grcanosa-alstom]

I do not have a reproducible example. It works in my local laptop but not on my CI agent. :(

I have downgraded to 1.2.0 as suggested and it also works in the CI agent so I will stick with that version for now.

[oyvindberg]

I got this now, in a VM which was running a surprisingly old version of ubuntu.

$ cat /etc/issue
Ubuntu 16.04.4 LTS

$ sbt importer/assembly
[info] welcome to sbt 1.9.6 (Private Build Java 1.8.0_292)
[info] loading settings for project tso-build from plugins.sbt ...
[info] loading project definition from /home/tso/tso/project
[info] compiling 1 Scala source to /home/tso/tso/project/target/scala-2.12/sbt-1.0/classes ...
[info] loading settings for project root from build.sbt ...
[info] resolving key references (11679 settings) ...
[info] set current project to converter-root (in build file:/home/tso/tso/)
<snip>
[info] compiling 18 Scala sources to /home/tso/tso/importer/target/scala-2.12/classes ...
[info] 48 file(s) merged using strategy 'Rename' (Run the task at debug level to see the details)
[info] 218 file(s) merged using strategy 'Discard' (Run the task at debug level to see the details)
[info] 4 file(s) merged using strategy 'First' (Run the task at debug level to see the details)
[info] 2 file(s) merged using strategy 'Concat' (Run the task at debug level to see the details)
[info] 2 file(s) merged using strategy 'FilterDistinctLines' (Run the task at debug level to see the details)
[info] 10 file(s) merged using strategy 'Deduplicate' (Run the task at debug level to see the details)
[error] java.nio.file.InvalidPathException: Malformed input or input contains unmappable characters: shapeless/$tilde$qmark$greater$λ.class
[error] 	at sun.nio.fs.UnixPath.encode(UnixPath.java:147)
[error] 	at sun.nio.fs.UnixPath.<init>(UnixPath.java:71)
[error] 	at sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:281)
[error] 	at java.nio.file.Paths.get(Paths.java:84)
[error] 	at sbtassembly.Assembly$.$anonfun$reportConflictsMissedByTheMerge$1(Assembly.scala:652)
[error] 	at sbtassembly.Assembly$.$anonfun$reportConflictsMissedByTheMerge$4(Assembly.scala:661)
[error] 	at scala.collection.TraversableLike.$anonfun$flatMap$1(TraversableLike.scala:293)
[error] 	at scala.collection.Iterator.foreach(Iterator.scala:943)
[error] 	at scala.collection.Iterator.foreach$(Iterator.scala:943)
[error] 	at scala.collection.AbstractIterator.foreach(Iterator.scala:1431)
[error] 	at scala.collection.IterableLike.foreach(IterableLike.scala:74)
[error] 	at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
[error] 	at scala.collection.AbstractIterable.foreach(Iterable.scala:56)
[error] 	at scala.collection.TraversableLike.flatMap(TraversableLike.scala:293)
[error] 	at scala.collection.TraversableLike.flatMap$(TraversableLike.scala:290)
[error] 	at scala.collection.AbstractTraversable.flatMap(Traversable.scala:108)
[error] 	at sbtassembly.Assembly$.$anonfun$reportConflictsMissedByTheMerge$3(Assembly.scala:660)
[error] 	at scala.collection.parallel.AugmentedIterableIterator.flatmap2combiner(RemainsIterator.scala:133)
[error] 	at scala.collection.parallel.AugmentedIterableIterator.flatmap2combiner$(RemainsIterator.scala:130)
[error] 	at scala.collection.parallel.immutable.ParVector$ParVectorIterator.flatmap2combiner(ParVector.scala:66)
[error] 	at scala.collection.parallel.ParIterableLike$FlatMap.leaf(ParIterableLike.scala:1082)
[error] 	at scala.collection.parallel.Task.$anonfun$tryLeaf$1(Tasks.scala:53)
[error] 	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
[error] 	at scala.util.control.Breaks$$anon$1.catchBreak(Breaks.scala:67)
[error] 	at scala.collection.parallel.Task.tryLeaf(Tasks.scala:56)
[error] 	at scala.collection.parallel.Task.tryLeaf$(Tasks.scala:50)
[error] 	at scala.collection.parallel.ParIterableLike$FlatMap.tryLeaf(ParIterableLike.scala:1078)
[error] 	at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.internal(Tasks.scala:170)
[error] 	at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.internal$(Tasks.scala:157)
[error] 	at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.internal(Tasks.scala:440)
[error] 	at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.compute(Tasks.scala:150)
[error] 	at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.compute$(Tasks.scala:149)
[error] 	at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:440)
[error] 	at java.util.concurrent.RecursiveAction.exec(RecursiveAction.java:189)
[error] 	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
[error] 	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
[error] 	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
[error] 	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
[error] (importer / assembly) java.nio.file.InvalidPathException: Malformed input or input contains unmappable characters: shapeless/$tilde$qmark$greater$λ.class
[error] Total time: 68 s (01:08), completed Sep 17, 2023 11:40:32 PM

The λ symbol is shown in my terminal as ?, don't know if that means something.

It's this project and commit ScalablyTyped/Converter@9ecf062
That's not the commit which introduced the problem, but one that contains it. I tried to go a bit back in time, so it had been broken for a while apparently.

[ncaq]

My workaround in Dockerfile.

ENV LANG=C.UTF-8
ENV LC_ALL=C.UTF-8

[LyndonArmitage]

I was able to workaround this issue using the environment variables:

LANG=C.UTF-8
LC_ALL=C.UTF-8

In my build pipeline. Adding -Dsun.jnu.encoding=UTF-8 to my .sbtopts did not help.

I don't have a reproducible example as it works on my local machine but not the build agent we use, similar to others.
I think you might be able to build one though: using Docker you could create an sbt instance with a specific language set and have it build a simple project that include the shapeless library and an assembly for it.

Unsure if the stack trace will help, but the docker image my build is using is: sbtscala/scala-sbt:graalvm-ce-22.3.3-b1-java17_1.10.3_2.13.15

That's:

• sbt 1.10.3
• Scala 2.13.15
• Java 17.
• With sbt-assembly version 2.3.0