Can't pull in JSON from S3 connector

I'm trying to pull in a few json files from the S3 connector and one of the files throws up this error:

 Failed parsing JSON source: JsonReader at line 1 column 24976613 to Json

 

The other two json files ran successfully and are very similar to the one that isn't pulling. I'm unsure on how to troubleshoot this.

Comments

  • GrantSmith
    GrantSmith Indiana 🔴

    Hi @user14700 

    Have you tried downloading your file manually and then using an online JSON parser to see if there's a formatting issue with your file?

  • Haven't thought of that, i'll give that a try right now!

  • I tried an online JSON parser and it was formatted correctly so I don't think there's an issue there. Is there a limit how many rows I can pull in? It's around 437k rows with only like 5 columns.

  • jaeW_at_Onyx
    jaeW_at_Onyx Budapest / Portland, OR 🟤

    maybe try uploading a subset of the data?

    do you have clean key-value pairs, or do your values occasionally contain arrays? 

  • Just tried uploading it with different sizes. I'm starting to think there is a cap because I uploaded 3/4 the size and it worked normally but when I removed a few dozen rows, I get the same error. It also says Updated:Invalid date on the top however there's no date field in the JSON file.

  • GrantSmith
    GrantSmith Indiana 🔴

    The other test you can do is test the final 1/4 in a separate file to confirm is a size issue and not a record formatting issue in the file.