I have 10 GB of data and I want to run a python script on that. But I am getting Out of memory issue because of the size of data.
I want to limit the number of records from the input dataset. How to do it? Thanks in advance for suggestions and solutions.
Are you using the Magic Transformation: ETL? Or how are you running python scripts?
If you are using the ETL, what about using the Filter and just choose a subset of your data based on any condition you'd prefer?