php - How do I keep a local copy of a large JSON file updated? -

php - How do I keep a local copy of a large JSON file updated? -

i'm writing php-based website, , requires json table other website. it's relatively big file hardly ever gets updated, preferably i'd maintain local re-create , utilize that, rather request server same file.

but need latest version of json file, otherwise lot of links site break.

i'm new web-development i'm not sure how approach this. i've read on things redis , memcaching, i'm not sure if that's i'm looking or how working.

how can maintain local version of json file date, or there improve approach?

normally in situations these request updated re-create of file every x hours , store on server. in pracice: on every page load check how old file (filemtime()); if it's older x hours, download 1 time again , replace contents of cached files. x, number of hours, has adjusted accordingly frequency update original document. example, if know document updated every 6 hours, may want refresh every 2 hours (x=2), example.

you should prepared have outdated cache, however: @ moment cached file can x hours (e.g. if download file , webmasters replace 1 min later, cached file outdated x hours less minute).

there ways can improve process:

if file big, downloading while preparing web page in response visitor can bad idea. indeed, visitors hate waiting, , every ms more takes generate page more visitor go away , lose interest. there studies that (ironically, page took me few seconds load :) - , in meanwhile went tab type). solution: have background job periodically refreshes cached file. can cron; if don't have access cron, there other ways emulate in php. example, see the guys @ techcrunch came with. instead of downloading entire file every time, (if remote server supports it) first create request see if file has changed. that's possible if-modified-since header. see illustration this question. way create head http request , check lastly modified time. note not servers may back upwards these tricks (especially if remote server generating file dynamically). if supported, however, can decrease time interval between each request (the x above), since won't downloading whole file every time.

php json