Can't pull in JSON from S3 connector

I'm trying to pull in a few json files from the S3 connector and one of the files throws up this error:

 Failed parsing JSON source: JsonReader at line 1 column 24976613 to Json

 

The other two json files ran successfully and are very similar to the one that isn't pulling. I'm unsure on how to troubleshoot this.

Comments

  • GrantSmith
    GrantSmith Indiana 🔵

    Hi @user14700 

    Have you tried downloading your file manually and then using an online JSON parser to see if there's a formatting issue with your file?

  • Haven't thought of that, i'll give that a try right now!

  • I tried an online JSON parser and it was formatted correctly so I don't think there's an issue there. Is there a limit how many rows I can pull in? It's around 437k rows with only like 5 columns.

  • jaeW_at_Onyx
    jaeW_at_Onyx Budapest / Portland, OR 🔵

    maybe try uploading a subset of the data?

    do you have clean key-value pairs, or do your values occasionally contain arrays? 

  • Just tried uploading it with different sizes. I'm starting to think there is a cap because I uploaded 3/4 the size and it worked normally but when I removed a few dozen rows, I get the same error. It also says Updated:Invalid date on the top however there's no date field in the JSON file.

  • GrantSmith
    GrantSmith Indiana 🔵

    The other test you can do is test the final 1/4 in a separate file to confirm is a size issue and not a record formatting issue in the file.

Sign In or Register to comment.

Hey, Stranger!

It looks like you're new here. Those who sign in get access to engage with even MORE fire content. To get involved, click one of these buttons!