Re: Db2 and clob columns

lastly,

when testing...

is the best way to confirm no padded spaces being stored is to...

clear table
run a few transactions to store the rows
divide table size by row count?

(should not show storage up to 10mb (for the 2 5mb clob columns))... right?

Jay

On Tue, Apr 16, 2024 at 12:08 PM Jon Paris <jon.paris@xxxxxxxxxxxxxx>
wrote:

You missed the

g_clob_data = i_5000000;

bit. But otherwise yes.


Jon P.

On Apr 16, 2024, at 11:12 AM, Jay Vaughn <jeffersonvaughn@xxxxxxxxx>

wrote:

ok that is nearly the code I already had with the exception of...

i_5000000 varchar(5242880) const;

and

g_clob_len = %len(i_5000000);

Jon, are you saying this is really the only change I really need?
Just wanting to confirm.

thanks

Jay



On Tue, Apr 16, 2024 at 11:02 AM Jon Paris <jon.paris@xxxxxxxxxxxxxx>

wrote:

As Charles has pointed out the Trims here are pretty much a waste of

time

if the varchar was loaded correctly (i.e. via a %TrimR).

Also in the rare event that the caller _wanted_ trailing spaces and

had

set the field up accordingly then this code would remove them.

dcl-pi
i_5000000 varchar(5242880) CONST;
end-pi;

dcl-s g_clob sqltype(clob:5242880) static;

g_clob_data = %TRIMR(i_5000000);
g_clob_len = %len(%TRIMR(i_5000000));

In fact the code should just be:

dcl-pi
i_5000000 varchar(5242880) CONST;
end-pi;

dcl-s g_clob sqltype(clob:5242880) static;

g_clob_data = i_5000000; // Trim is pointless as the compiler will

just

add back blanks to pad g_clob_data anyway
g_clob_len = %len(i_5000000); // Again the trim does nothing except
waste time


Jon P.

On Apr 16, 2024, at 10:50 AM, Charles Wilt <charles.wilt@xxxxxxxxx>

wrote:

Thinking about it...

While the %TRIMR() should be a no-op if the data is already

trimmed...

I don't think %LEN(%TRIMR()) would be...

Depending on how often this is called, I might get rid of the

%TRIMR()...or

only do them if the %LEN() is larger than the 64k expected.

But the change to CONST and the use of STATIC will greatly increase

the

performance compared to the original.

Charles

On Tue, Apr 16, 2024 at 8:21 AM Charles Wilt <charles.wilt@xxxxxxxxx

wrote:

Since g_clob_data is CHAR(5242880)

The %LEN(g_clob_data) is always 5242880, so yep you're writing out

blanks.

You should use %LEN(i_5000000);

But even there, if the data is passed in with trailing blanks, then

yep

you're getting 5MB written out.

A quick fix would be to add OPTIONS(*TRIM) to the parameter, but

that

would require a recompile of callers.

You could also add %TRIMR(), but you'd end up doing so twice...

Personally, I'd prefer to ensure the data is trimmed at the origin

and

then pass in as CONST instead of VALUE instead of copying the value

to

the

stack.
Also, I'd define g_clob as STATIC, to keep from having it created

and

deleted (and INZ) for every call.

I might take a boots and suspenders approach, including a %TRIMR()

...

if

the data is already trimmed it should be a no-op.

dcl-pi
i_5000000 varchar(5242880) CONST;
end-pi;

dcl-s g_clob sqltype(clob:5242880) static;

g_clob_data = %TRIMR(i_5000000);
g_clob_len = %len(%TRIMR(i_5000000));

insert into table(myClob)
values (:g_clob);


Charles


On Tue, Apr 16, 2024 at 7:38 AM Jay Vaughn <

jeffersonvaughn@xxxxxxxxx

wrote:

we have a procedure...

dcl-pi
i_5000000 varchar(5242880) value;
end-pi;

dcl-s g_clob sqltype(clob:5242880) inz;

g_clob_data = i_5000000;
g_clob_len = %len(g_clob_data);

insert into table(myClob)
values (:g_clob);


thoughts?

Jay







On Tue, Apr 16, 2024 at 9:31 AM Charles Wilt <

charles.wilt@xxxxxxxxx

wrote:

If you have RPG code writing out the table, I'd look there first.

Make sure the RPG code is properly handling the varchar data.

Charles

On Tue, Apr 16, 2024 at 7:00 AM Jay Vaughn <

jeffersonvaughn@xxxxxxxxx

wrote:

So Charles,

We have a table developed to hold a request clob (5mb) and a

response

clob

(5mb) ONLY for storage and auditing purposes.
Not to run queries against.

We have been finding out that some off platform load tests are

being

performed and even though the api calls to put rows in this table

should

have 1mb or less requests/responses, we are maxing out the DASD

on

that

test box and this table ends up with 821G of data.

Again our requests/responses should be well under 1mb each.

So is this table just a bad idea or are we simply over dominating

the

box

with our load test volume?

Next time this happens I really want to look at the rows and see

what

these

columns have in them.

thanks

Jay








On Mon, Apr 15, 2024 at 6:51 PM Jay Vaughn <

jeffersonvaughn@xxxxxxxxx

wrote:

Thanks Charles.

I’lllook into the allocate but big relief on the storage

concern.

Jay

On Apr 15, 2024, at 5:44 PM, Charles Wilt <

charles.wilt@xxxxxxxxx

wrote:

Nope...

variable length data is stored in the overflow section of the

table...

Downside of that is that when you read the row, it takes 2

I/Os.

One

for

the row space and one for the overflow.

If you were mostly writing less that 32k, I'd tell you to take

a

look

at

the ALLOCATE clause. That does cause space to be reserved in

the

standard

table space. Which wastes space, but then the data only needs

1

I/O

to

be

read.

Charles

On Mon, Apr 15, 2024 at 2:20 PM Jay Vaughn <

jeffersonvaughn@xxxxxxxxx

wrote:

Let’s say I have a table with a 5mb clob column.

Most of my transactions that write to the table mostly consist

of

64k

and

less chars written to the clob but we want to reserve the much

higher

storage allocation for when we need it

When the 64k chars are written to that clob on the row, the

entire

5mb

of

storage is not written to disk is it??

Tia

Jay

Sent from my iPhone
--
This is the Midrange Systems Technical Discussion (MIDRANGE-L)

mailing

list

To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxxxxxxxxxx for any

subscription

related

questions.

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L)

mailing

list

To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxxxxxxxxxx for any

subscription

related questions.

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L)

mailing

list

To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxxxxxxxxxx for any subscription

related

questions.

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L)

mailing

list

To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxxxxxxxxxx for any subscription

related

questions.

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L)

mailing

list
To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxxxxxxxxxx for any subscription

related

questions.

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L)

mailing

list

To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxxxxxxxxxx for any subscription

related questions.

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing

list

To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxxxxxxxxxx for any subscription

related

questions.

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing

list

To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxxxxxxxxxx for any subscription

related questions.

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing

list

To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxxxxxxxxxx for any subscription related
questions.

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list
To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxxxxxxxxxx for any subscription related
questions.