iWay Big Data Integrator User's Guide > Defining Run Configurations > Defining Run Configurations for a Spark Deployment

Defining Run Configurations for a Spark Deployment

Custom Spark code can be run with iWay Big Data Integrator (iBDI) using the iWay Big Data Integrator Spark Deploy option.

Perform the following steps to define a run configuration for a Spark deployment:

  1. Right-click iWay Big Data Integrator Spark Deploy in the left pane of the Run Configurations dialog and select New from the context menu.

    Note: If an existing run configuration is available, you can right-click that instance and select Duplicate from the context menu. You can then modify only those parameter values that are required without having to configure all of the parameters.

    A new run/build configuration pane opens.

  2. Enter a descriptive name for the new Spark deployment run configuration in the Name field. Avoid using dashes or slash characters in the name.
  3. Click Browse.

    Navigate to the location of a Scala source file or Spark .jar file.

  4. Click OK.
  5. Enter the following connection information:
    • Host Name
    • User Name
    • Password
  6. In the Deployment Location field, enter the path to the iBDI provision location where the run time object will be deployed.
  7. Click Apply to save the information that you specified.

    To clear all of the information that you specified in this dialog, click Revert.

  8. Click Run to run the Spark deployment.

    The Console pane opens and displays the results for this deployment.