Apache Nifi | Aviation Data Flow | RESTful API with InvokeHTTP Processor | Part 1

Working with RESTful API’s in Apache Nifi is a great way to consume data.

Support the channel by Subscribing!

SUPPORT THE CHANNEL

FIND ME ON THE WEB

Resources:

What is Apache NiFi?

Put simply NiFi was built to automate the flow of data between systems. While the term 'dataflow' is used in a variety of contexts, we use it here to mean the automated and managed flow of information between systems. This problem space has been around ever since enterprises had more than one system, where some of the systems created data and some of the systems consumed data.

[StevenKoon]

Working with API’s in Nifi can be pretty simple and powerful at the same time. If you like the video, then consider subscribing to the channel and getting notified about more video’s.

Let me know how you liked the video in the comments.

[debarghyadasgupta]

Thanks Steven. Good one to kick start the journey. Looking forward to the series. Appreciate for putting this for the community.

[tw]

You are looking good old man!! We served together in Hawaii.

[bonnydavis]

That was very informative. Thank you Sir.

[arpithamadduri]

@Steven Koon Thank you for the informative video. I am trying to build something I believe would work, I am struck at the configuring proxy. Could you share an example with api that uses Username, Password, and Proxy?

[PetrenkoAndrii]

Nice guide! Thanks!
You said that getting data thru the https:// needs just writing "s" in the link. I've tried using https link, but this does not work.
I figured that SSL Context Service must be set up first. Can you, please, make short video on SSL Context Service for NiFi to use https links?

[azizutkuozdemir]

thanks a lot for sharing this content, nice kick start

[rickrofe]

Great overview, thanks for taking the time liked and subscribed....

[tanababa]

Thank you Steven for the great tutorial.
I was wondering if there is also a tutorial for creating a CSV or writing to a DB the data we got from the API

[bigoper]

How would you tackle this if you need to paginate using the limit/offset/etc...? thank you!

[dubenelevy]

Hi Steven, I just installed nifi and started to follow this video. When I go to the list queue and tried and view content. The "original" and "formatted" do not contain any data and it says "No viewer is registered for this content type." I also cannot change the content type to json.

[skawalker]

thank you for sharing this guide Steven! how do you go about in using the invokeHTTP processor multiple times where the API parameter will have different value for each run - a good example is that I notice in the avaiationstack API documentation there is an "offset" parameter I assumed it is used to fetch data in batch. thanks in advance!

[vinaynayak]

Very helpful! Thinking about how to augment this flow to incorporate re-generation of APIKey. So when the InvokeHTTP fails because of expired APIKey how does one know that the failure is due to the APIKey and the flow needs to be regenerate it?

[xxPowrslave]

Thanks a bunch for this! I'm going to check out your other videos. Liked and subscribed :) show us some other cool use-cases in NiFi!

[ChinnuReddy-blng]

Hi Steven, I have tried to integrate Veeva vault to Apache Nifi with InvokeHTTP Processor. To fetch the session id, what are the parameters have to use?

[venkatapenumatsa]

Hi Steve, Thanks for the tutorial. Nifi is installed in your system? or do you use any cloud instances like AWS etc? Do you have any tutorials as to how to install Nifi in AWS?

[NotaThing]

Nice video… i see you talking about dynamically changing access_key from an upstream flow. I’m just starting. Was wanting to do something like that I believe.

I want to add an attribute based on the flowfile content. I get a response from a DB with columns I want to use in API calls. maybe airports is part of response and I have a row as a flowfile. Is there a simple way to pass attributes from flowfiles? could I add a property to ExecuteSQL that reads a column value? Do I need to add another processor like updateAttribute? Do you have any videos covering something like this? Thank you 🙏

*Edit:
Was able to get what I needed with ConvertAvroToJSON and then EvaluateJsonPath

[DhanbadWalaOfficial]

How do we send an email alert from nifi what should be flow can you name some exact processor?

[gustyflores]

Thank you Steven! how can I see the url that were concatenated during the execution (in order to validate if I am concatenating properly) ?

[abdelrahimahmad]

Steven, you are great. We will support you. do you have a website!?