|
midrange.com
Many of you helped me through the evolution of this, so I wanted to share the
final solution with the list!
My goal in this project was to implement caching of my Net.Data website. A
secondary goal was to be able to generate or regenerate pages "on demand."
Originally I had hoped to call a CL program from within Net.Data to call a
second instance of Net.Data to generate the page into the cache. It seems it
just isn't possible to call it twice in the same job, so I reverted to
submitting the second call to batch.
I also decided that I was wasting my time moving the cached page to the IFS and
instead am serving from a PF. That had a side-effect of eliminating character
translation issues when data was moved to the IFS, and makes it easy to have
the first job to know when the page is available; it just tries to
@DTW_READFILE, and as soon as the lock on that page is released, it reads it.
Here's how it was implemented:
ENTRY POINT
HTTP traffic is directed to this Net.Data page using Apache config:
ScriptAliasMatch ^/$ /QSYS.LIB/DISTV2.LIB/DB2WWW.PGM/Cache.mac/Cache
ScriptAlias /index.html /QSYS.LIB/DISTV2.LIB/DB2WWW.PGM/Cache.mac/Cache
%HTML(Cache) {
%{ Generate filename of cached page based on K, SearchTyp, and SearchVal
The cached pages are kept in a PF, so the filename is limited to 10 chars.
This logic takes all possible input variables and decides what the cached
filename is.
%}
%{ Default is INDEX %}
@DTW_ASSIGN(CacheFileName,"INDEX")
%IF(SearchTyp&&SearchVal)
@DTW_ASSIGN(CacheFileName,"S$(SearchTyp)$(SearchVal)")
%ELIF(K)
%IF(@DTW_rSUBSTR(K,"1","1")=="*")
@DTW_ASSIGN(CacheFileName,@DTW_rSUBSTR(K,"2",@DTW_rEVAL("@DTW_rLENGTH(K)-1")))
%ELSE
@DTW_ASSIGN(CacheFileName,"K$(K)")
%ENDIF
%ENDIF
@DTW_ASSIGN(CacheMbrName,CacheFileName)
@DTW_CONCAT(CacheFileName,".MBR",CacheFileName)
%{ Perform any work requested and blow away old cached page %}
%IF(BBETXT)
@SaveFeedback(K, BBEBBS, BBENAM, BBETTL, BBETXT, STFEMLPUB)
%ENDIF
%{ Check for existing cached copy of requested page
In my case, there are only two ways that a page gets refreshed:
1: I can force it to by adding Force=X to the URL
2: If the user just left feedback on a review (BBETXT is populated)
I could use DTWF_EXISTS to see if it exists yet in the cache, but at this
time I limit it
to displaying or regenerated pages that are already in the cache. I will
add that once I can
validate that the page being requested is "real".
%}
%IF(BBETXT||Force)
@SubmitCacheItem("K=$(K)&SearchTyp=$(SearchTyp)&SearchVal=$(SearchVal)",CacheMbrName)
@SubmitCG()
%ENDIF
%{ Serve page from cache
If the page isn't in the cache by now, then I'm happy to throw a 404.
Read it and display it. The side-effect is that if the page is being
regenerated right now,
@DTWF_READFILE will be LCKW until it's completed. Note that I found
@DTWF_EXISTS and @DTWF_READFILE
to be very flaky when dealing with files on the IFS, randomly returning
error 2000 until the
HTTP server was restarted.
FFI_PATH is QSYS.LIB/DISTV2.LIB/CACHE.FILE, so I just need a .MBR name to
read it.
%}
@DTWF_READFILE(CacheFileName,CachePage)
$(CachePage)
%{ Log hit
This is how I track my hits. I'm sure you're excited.
%}
%IF(K=="*APPLY"||K=="*SEARCH"||K=="*STAFF")
@LogHitDsc(K)
%ELIF(K)
@LogHitK(K)
%ELSE
@LogHitDsc("*HOME")
%ENDIF
%}
PAGE GENERATION
If a page needs to be generated, @SubmitCacheItem() and @SubmitCG() are called.
%FUNCTION(DTW_SQL) SubmitCacheItem(IN InQryStr, InMbrNam) {
Insert into $(DB).CACHEQUE
Values('$(InQryStr)','$(InMbrNam)')
With NC
%{ This should check for dupes before inserting... but it doesn't yet %}
%}
%FUNCTION(DTW_DIRECTCALL) SubmitCG() {
%EXEC {
/QSYS.LIB/DISTV2.LIB/SUBMITCG.PGM
%}
%}
Up to this point, everything has been using the same Net.Data library, INI
file, and DB2WWW.
Here's program SUBMITCG:
PGM
SBMJOB CMD(CALL PGM(DISTV2/CGPRCQUE)) JOB(CGWEBREQ)
DLYJOB DLY(4)
ENDPGM
The purpose of the DLYJOB is to ensure that the submitted job has established a
lock on the cache file member before handing control back to Net.Data which is
going to try to @DTWF_READFILE immediately. Again, the nice part about serving
from a PF is that @DTWF_READFILE will wait for the lock to be freed, meaning I
don't need any further communication with CGPRCQUE which has just been
submitted to batch.
And finally, here's program CGPRCQUE:
PGM
DCLF FILE(DISTV2/CACHEQUE)
/* setup main environment for net.data */
CHGCURDIR DIR('/QSYS.LIB/DISTCGV2.LIB')
ADDENVVAR ENVVAR(PATH_INFO) +
VALUE('CACHEGEN.MAC/CACHEGENINDEX') +
REPLACE(*YES)
LOOP: RCVF
MONMSG MSGID(CPF0864) EXEC(GOTO CMDLBL(END))
CLRPFM FILE(DISTV2/CACHE) MBR(&MBR)
MONMSG MSGID(CPF3141) EXEC(ADDPFM +
FILE(DISTV2/CACHE) MBR(&MBR))
OVRDBF FILE(STDOUT) TOFILE(DISTV2/CACHE) MBR(&MBR)
/* Query_String is the only thing Net.Data needs to know that will change for
each page */
ADDENVVAR ENVVAR(QUERY_STRING) VALUE(&QRYSTR) REPLACE(*YES)
CALL PGM(DISTCGV2/DB2WWW)
DLTOVR FILE(STDOUT)
GOTO CMDLBL(LOOP)
END: CLRPFM FILE(DISTV2/CACHEQUE)
ENDPGM
Note that this program calls Net.Data in the DISTCGV2 library, different from
before, allowing for different configuration and better security. It runs from
a single-threaded jobq ensuring that the active job processes all pending
requests. Additional jobs submitted from a simultaneous request will be queued
and if they find CACHEQUE empty, will simply end. It's not industrial-strength,
but it works.
The Cachegen.mac/CacheGenIndex HTML block is non-the-wiser, generates the page
as it always has, but the STDOUT override places the page into the cache. Once
this completes, the lock is free, and Net.Data serves the page. The only change
to that macro was adding DTW_PRINT_HEADER="NO".
So the question is, is it fast? After all, this is all about caching. Well,
judge for yourself. www.Distortion.us. And that's on a 510-2143. Add Force=X to
the URL to see how long it used to take...
Thanks again to everyone who helped in this little project. I hope this
information will help someone else down the road. And if you have any
suggestions on improving this, I'd love to hear them!
As an Amazon Associate we earn from qualifying purchases.
This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].
Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.
