So let me start off by saying that as of 3 hours ago, I knew nothing about Python. I've been learning as I go. I've been able to connect to the API I am working with, create and refresh oAuth2.0 tokens and make successful requests. I've saved those requests to .json files.
The next step was to loop my request and get ALL of the data (the API has a limit = 100). I've come up with a way to make that work, however, when it comes to writing to the .json file... I have some issues.
It writes fine, and it appends each step of the loop no problem. The issue is, it's adding the open and close portions of the .json every time. So when I try to use the data, I just get errors saying there's unexpected data after the file should have ended.
So my question is... how can I gather all of these paginated results and write them to one properly formatted .json?
Here's my initial request for data:
# making the initial request
url = my_url
headers = {
"Authorization": token_type + SPACE + access_token,
"Content-Type": "application/json"
}
response = request("GET", url, headers=headers)
# print(response.text)
nextsetup = response.json()
next = nextsetup['@attributes']['next']
#write response to file
writeFile =open('items.json', 'w')
writeFile.write(response.text)
writeFile.close()
Here's my loop:
while next:
url = next
headers = {
"Authorization": token_type + SPACE + access_token,
"Content-Type": "application/json"
}
responseloop = request("GET", url, headers=headers)
nextsetup = responseloop.json()
next = nextsetup['@attributes']['next']
print("page: ", page) #printing just to show me the loop is still running
page = page+1
#write response to file
writeFile =open('C:/Users/apedd/Documents/LightspeedAPI/items.json', 'a')
writeFile.write(response.text)
writeFile.close()
I know it's messy, again I'm learning and I've used extra variables and steps I probably don't need as I have been trying other methods to get the output I need. This code "works" in that it gets all of the data and it saves it all... but it's adding things together as if I took several .json files and just copy/pasted the plain text into one big .json. Giving me error:
SyntaxError: JSON.parse: unexpected non-whitespace character after JSON data at line 1 column 75498 of the JSON data