How to: |
This section describes how to configure Apache NiFi. You can specify Apache NiFi as a streaming source stage in a Pipeline configuration.
For more information on installing Apache NiFi, see the iWay Big Data Integrator Provisioning Guide.
http://hostname:port/nifi
where:
Is where Apache NiFi is installed and running.
Is the port number that was assigned to be used by Apache NiFI during the installation process. For more information, see the iWay Big Data Integrator Provisioning Guide.
The Apache NiFi web-based user interface (UI) opens.
The Processor Details dialog opens, as shown in the following image.
Property |
Value |
---|---|
Command |
tail |
Command Arguments |
/opt/nifi/nifi-1.1.2/logs/nifi-apps.log |
Batch Duration |
20sec |
Redirect Error Stream |
false |
A connection object will be created to complete the FlowFile, as shown in the following image.
Specify the URL of your Apache NiFi instance (the server where it is running) and the name of the Output Port object that was created for Spark (nifi2bdi).
home/myuserid/bdipipelines/mynifi/
cd /work
cp ${HDM_HOME}/lib/nifi-*.jar .
Note: Ensure that there is a space between jar and the period character (.)
./run.sh
The NiFi streaming should begin.
Note: If an Unsupported Major.Minor Version exception should occur, then wrong Apache NiFi libraries have been deployed for the runtime Java version.
Check the installed version of Java on the edge node by typing which java at the command line to show the folder where Java is installed.
Check the version of Java by typing java -version.