Loading Random Objects From A Huge (>200MB) Array File Without Loading The Entire Array

I am building a static Angular website whose data is based on 1000 random objects from a huge array that contains more than 200,000 JSON objects. I am storing the huge array of JSO

Solution 1:

You could attempt to download only the first partial chunk of the file and try to use that.

https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests

You would use the Angular's HTTP service to make a GET request, but would add Range: bytes=0-1023 to the headers. Where 0-1023 defines the number of bytes you would like to fetch (using a start and end range).

If the server supports this feature, then you'll have to clean up the string JSON that is fetched from the server. Since the JSON will be cut-off before it's complete.

Without me knowing what the structure of the JSON is. It's difficult to say if this is a practical solution, but if it's an array of JSON objects. You could try to find a byte range that ends on an array separator , character and just append ] to the end of the string.

If you need random parts of the file. You would make multiple GET requests for different partial chunks, and then try to clean the JSON so that it's valid.