Class HadoopUploadModule

  • All Implemented Interfaces:
    Module

    public class HadoopUploadModule
    extends UploadModule
    This class define a module for Hadoop file uploading.
    Since:
    1.0
    Author:
    Laurent Jourdren
    • Constructor Detail

      • HadoopUploadModule

        public HadoopUploadModule​(DataFile dest,
                                  org.apache.hadoop.conf.Configuration conf)
        Public constructor.
        Parameters:
        dest - destination of the files to upload
        conf - Hadoop configuration
    • Method Detail

      • getUploadedDataFile

        protected DataFile getUploadedDataFile​(DataFile file)
                                        throws IOException
        Description copied from class: UploadModule
        Generate the DataFile Object for the uploaded DataFile
        Specified by:
        getUploadedDataFile in class UploadModule
        Parameters:
        file - DataFile to upload
        Returns:
        a new DataFile object with the path to the upload DataFile
        Throws:
        IOException - if an error occurs while creating the result DataFile
      • getUploadedDataFile

        protected DataFile getUploadedDataFile​(DataFile file,
                                               Step step,
                                               Sample sample,
                                               String portName,
                                               DataFormat format,
                                               int fileIndex)
                                        throws IOException
        Description copied from class: UploadModule
        Generate the DataFile Object for the uploaded DataFile
        Specified by:
        getUploadedDataFile in class UploadModule
        Parameters:
        file - DataFile to upload
        sample - the sample for the source
        portName - the port name
        format - the format of the file to upload
        fileIndex - file index for multifile data
        Returns:
        a new DataFile object with the path to the upload DataFile
        Throws:
        IOException - if an error occurs while creating the result DataFile