How to Query AWS Athena from a Lambda Function | Step by Step Tutorial

In this video, I show you how to submit an Athena query and retrieve the results from a Lambda Function.

📚 MY RECOMMENDED READING LIST FOR SOFTWARE DEVELOPERS📚

🎙 MY RECORDING EQUIPMENT 🎙

💻 MY DESKTOP EQUIPMENT 💻

📚 References:

☁Topics covered include:
Lambda Athena Query
Lambda Athena IAM Permissions
Lambda Athena Query Submission
Lambda Athena Query Results

🌎 Find me here:

#AWS
#Serverless
#Lambda
#Athena
#CloudComputing

[AbdurrahmanKocukcu]

Pleasant and concise walkthrough. Thanks :)

[Spaz]

Quality video with quality information! I'm going to be looking at more of your videos!

[dojocoding]

May be we use async await instead of sleep ?

[withtheengineer-hamza-]

Was very
helpful, Thanks, the IAM permissions was to important too be detailed I guess

[manideepj]

Please provide the IAM roles used in this code deploy. If it provided in any cloud formation script can be appreciated

[thewisearchitect]

Very nice videos. Thanks for all the tutorials. I do have one question though.... Imagine we have a DynamoDB table which can get Inserts as well as Updates several times a day. We want to store all the inserts and updates (with historic versions of the same record over time) in S3. What is the best pattern that we should use to store these records in S3. Shall we always treat DB Updates as Inserts to S3. Do we also need to structure the S3 in hive-style folders hierarchy (e.g., date wise folders)? Will AWS Glue and Athena be able to query this data easily if the S3 files are placed in pre-structured date-organised folders? For instance, imagine a Customer's record underwent 3 changes at three different dates. We will end up saving his/her data within three different JSON/Parquet files and within different folders (due to different dates). Will Athena be able to return all versions of this Customer record if needed? Apologies for the long question.

[-shivanisarikonda]

Hi thanks for the video. I am working on this for my intern project and I am not able to view the query result csv in the output bucket passed in the lambda code. Isn't the query result csv should be in that bucket?

[ovapny]

Hi, don't you need to add some permissions for the S3 bucket?

[christojose]

Thank you so much for the video. Great learning opportunity for me!! Can we give a custom name for the resulting CSV generated by the Athena query? Currently, it is like Thanks in advance.

[davidalexander]

why waiting for the query to be resolved in the lambda instead of triggering a s3 event for the result bucket ?

[carlosmaturanapardo]

How can I transform the results to json format?

[mdhruv]

Can we also add a video of how to use aws cli ssm and putty to ssh into an ec2 . I have seen how to get to it from the aws console but I cannot get to it using aws cli using windows

[PLGARIS]

I have been trying to do the same, but I get a 403 when Athena tries to write to the output S3 bucket. The bucket itself allows all actions. Is there any access to S3 required to be configured?

[banusreemohankumar]

I can see you use BOTO3, is there a specific reason for python runtime preference over Java? Can you share some java code samples if possible.thanks !

[thanhnguyentrung]

Hi Bro, Please show source code of Role LambdaAthenaRole? 02:28

[charlotteh]

I've been struggling with finding a way to get a presigned url for an athena query output triggered by a lambda via API Gateway back as API response. Any chance you could make a video on that? :)

[universallocal]

Hi . Cant we create New databse and its tables from Boto3 in athena?. I am not finding any document of that .. On official Boto3 website also no mention of creating Tables and database

[skippy]

What is the name of the diagram software that you use at the beginning of this video?