|
The size of the file directly mirrors the size of the CLOB storage
(since the key data is measured in 100s of bytes vs. the CLOB measuring
in 100s of KB, the non-CLOB data is negligible). Note there are still
practical issues: the maximum size for a CLOB field is around 16MB; I
wouldn't store files larger than that in the database anyway. But for
anything smaller you're in good shape.
Generally speaking SAV performance is a lot faster. We actually had to
exclude the folders containing millions of files just to get backups to
complete. Saving a single 4GB file is much more efficient than saving a
million 4KB files.
Note that in order to use CLOBs you need commitment control which means
you have to journal the files. In most applications, the journals are
not useful so you can just delete the receivers as they roll over.
Joe
Joe, have you seen any differences in the amount of storage each record takes?
I can understand the benefits you see with access, and such. Any differences with saves, other than SAVSECDTA?
--
Bryan
On May 17, 2013, at 9:34 PM, Joe Pluta <lists@xxxxxxxxxxxxxxxxx> wrote:
I just got done dealing with this issue. I had two completely different
systems using the IFS for XML document storage (one for order
management, the other for EDI). It turns out that the IFS in particular
and the IBM i in general really aren't set up for handling millions of
documents.
Two issues. First is that any directory over 16000 entries begins to
fail on most QShell or PASE commands, although the native commands such
as MOV and DEL still work. But that does you no good when you want to
JAR up a directory with 150,000 files. Seconds, user profiles don't
like owning millions of objects. They just don't. They get bigger and
bigger and bigger and eventually your SAVSECDTA starts taking hours.
Not exactly the best situation.
So what I did is move everything back to the database. I use CLOBs and
store everything in a file. That allows me to easily manage the data
using traditional keys rather than crazy nested directory structures and
archival is as simple as copying records from one file to another. It's
a beautiful thing. As I said, I'm using it for XML but it's just as
applicable to PDFs or indeed any other stream file.
TIP: For XML you can use the XML data type rather than BLOB or CLOB. It
checks that the data is well formed and hopefully some day will also
allow PureXML querying like it does in DB2 LUW.
Joe
--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list
To post a message email: MIDRANGE-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/midrange-l.
I would imagine now you would start managing large database file backups vs IFS backups.
Now what does your nightly DB backup window look like ?
It's interesting that one of the strengths and great weaknesses of tools like MS Sharepoint is that they store all documents in the database.
Larger DB backup, less file backup or larger file backup less DB backup. I suppose either has merits.
In the case of Sharepoint there are 3rd party apps that allow the files to be stripped from the DBs back to the file system.
Personally rather than starting to dump blobs into the database, I would look at alternate object management methods on the IFS. We had one customer who was creating a new profile each month to mirror the year/month that docs were stored.
Or an better answer is probably to move the file objects to a SAN or NAS storage device.
Interestingly there's already a project on the docket at the customer Joel was talking about to move the docs to SAN. They just haven't done it yet.
Regards,
Richard Schoen
RJS Software Systems Inc.
Where Information Meets Innovation
Document Management, Workflow, Report Delivery, Forms and Business Intelligence
Email: richard@xxxxxxxxxxxxxxx
Web Site: http://www.rjssoftware.com
Tel: (952) 736-5800
Fax: (952) 736-5801
Toll Free: (888) RJSSOFT
As an Amazon Associate we earn from qualifying purchases.
This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].
Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.
midrange.com
