Mule - Copy the directory from HDFS

1
votes

I need to copy the directory (/tmp/xxx_files/xxx/Output) head containing sub folders and files from HDFS (Hadoop distributed file system). I'm using HDFS connector but it seems it does not support this.

It always getting an error like:

org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): Path is not a file: /tmp/xxx_files/xxx/Output/

I don't see any option is HDFS connector for copying the files/directories inside the path specified. It is always expecting file names to be copied.

Is it possible to copy a directory head containing sub-folders and files using the HDFS connector from MuleSoft?

1
mulehdfs
Could you please add some of your flow XML? - Attila

1 Answers

0
votes

As the technical documentation of the HSFS connector on the official MuleSoft website states, the code is hosted at the GitHub site of the connector:

The Anypoint Connector for the Hadoop Distributed File System (HDFS) is used as a bi-directional gateway between applications. Its source is stored at the HDFS Connector GitHub site.

What it does not state, that there is also a more detailed technical documentation available on the GitHub site.

Here you can also find different examples how to use the connector for basic file-system operations.

The links seem to be broken in the official MuleSoft documentation.

You can find the repository here:

https://github.com/mulesoft/mule-hadoop-connector

The operations are implemented in the HdfsOperations java class. (See also the FileSystemApiService class)

As you can see, the functionality you expect is not implemented. It is not supported out-of-the-box.

You can't copy a directory head containing sub folders and files from HDFS without any further effort using the HDFS connector.