|
First, I want to thank everyone who has contributed to this, you all--
have been immensely helpful with this issue! I've been able to make
some headway in this, and what I'm noticing is a drastic increase in
time it takes to read records. I knew that there would be a
performance hit, but 8 times as long!? Has anyone else noticed this?
I know this is off topic, but does anyone know if enabled TDE
(transparent data encryption) on the ASP level is any better? Or worse?
Thanks,
Nathan Hughes
Software Developer
601.499.2131 Office
280 Trace Colony Park
Ridgeland, MS 39157
-----Original Message-----
From: RPG400-L <rpg400-l-bounces@xxxxxxxxxxxxxxxxxx> On Behalf Of Jay
Vaughn
Sent: Sunday, March 29, 2020 8:37 PM
To: RPG programming on IBM i <rpg400-l@xxxxxxxxxxxxxxxxxx>
Subject: Re: FieldProc Error - column has returned invalid data on a
Zoned Numeric column
Vern,
No sql at all actually.
Though way underneath the covers the method does use the sql engine, I
believe.
The solutions comes by the way of Mark Waterbury. I myself, was in the
middle of developing an OA handler as well when he graciously shared
his alternative. It was truly a gift at the time as my entire projects
success was hinging on the ability to overcome this “predicament”.
Any pgm preceeding the usage of the encrypted file/field can be issued
an ovrdbf/opnqryf combination and way under the covers the sql engine
is invoked and processes the setll/read as it should by using the
clear txt value and not the encoded value. An incredibly simple yet
effective method. Thanks again Mark.
Anyhow this functionality is demonstrated in my download.
Jay
On Mar 29, 2020, at 8:01 PM, Vernon Hambergwrote:
<vhamberg@xxxxxxxxxxxxxxx>
waters. I did not at first understand that Mike was talking about
Hi Jay
I suspect my mention last week of the handler I wrote has muddied
the
creating the FIELDPROC. I don't remember the exact timing, but I think
we got through that early in this thread.
solution (very cool and generous, BTW) - do you require that people
So my question, related to your post about an end to end FIELDPROC
use embedded SQL for this, or SQL procedures? To avoid the problem
with a FIELDPROC over key fields? The vendor for who i wrote the
handler was faced with customers who could not afford to change everything to embedded SQL.
and programs.
Regards and stay well!
Vern
On 3/29/2020 6:06 PM, Jay Vaughn wrote:
The only thing Vern's OA handler is going to address is the side
effect that comes along with encrypted key fields paired with
native I/O combination of setll/read. Not to sell the handler
short. It was a huge accomplishment to have an handler developed
to translate any native i/o operation to the sql equivalent., so
not trying to sell it short... but as it relates to encryption and
field procedures, this is the area in which it is needed.
And this is applied to the application layer far after the field
procedure is created and attached to the file//field
jay
Jay
On Tue, Mar 24, 2020 at 8:45 AM Mike Jones
<mike.jones.sysdev@xxxxxxxxx>
wrote:
Hi Vernon,
It means field procedure programs can't run SQL statements. It
also means field procedure programs can't call other commands or
programs that run SQL statements. This even applies to calling
IBM commands
be a service program."
IBM's V7R3 SQL Reference manual page 1175, for the CREATE TABLE
statements for the FIELDPROC attribute says "Designates an
external-program-name as the field procedure exit routine for the
column. It must be an ILE program that does not contain SQL. It
cannot
applied,
I tried to make a field procedure program that would largely
self-deploy, where it would create a key store file (via IBM's
API), and generate / populate it with an encryption key, but
calling the API to create a key store file would not run. When I
looked into why, it was because IBM's API to create a key store
file runs SQL, and I found documentation that you can't do that.
I don't know why it is like that, just that it is.
Perhaps using an Open Access handler is a workaround, I don't know.
Those were my findings at the V7R3 level. I've not tried it on V7R4.
Mike
On Mon, Mar 23, 2020 at 9:39 PM Vernon Hamberg
<vhamberg@xxxxxxxxxxxxxxx>
wrote:
Mike - I'm not sure exactly what your statement means. If youdon't
mean that the programs themselves can't have embedded SQL, that's
something I don't know.
Now what I do know is that if a key field has a FIELDPROC added
to it, that the resulting order in RPG is probably incorrect. I
wrote an Open Access handler to turn all DISK IO in RPG into SQL
statements - only change in the RPG is to add the handler to the
F-spec. I wrote this for Townsend Security (this was publicly
announced, so no secrets being revealed here), I believe there is
a relationship now with Syncsort, if anyone has a need for this -
I'm not working for either company, so this is not a <verndor option>!!
Vern
On 3/23/2020 11:00 PM, Mike Jones wrote:
Nathan
Field proc programs don't yet allow the use of any SQL. They
also
runsyet support calling a program or command that directly or
indirectly
storeSQL statements. Example: you can't call IBM's API to create a
key
infile, from inside a field procedure program, because that API
runs SQL
but iits plumbing.
I've gotten all that encryption field procedure stuff to work
well,
itwas a PITA getting it all to work. I utilized code from athandful
least a
of sample programs.encryption
- AES 256-bit encryption
- DEKs (data encryption keys) stored in key store files.
- Using KEKs (key encrypting keys) to encrypt the DEKs (data
keys), also stored in key store files.retrieved
- Using master keys to encrypt the key store files.
- Using tokens with the encryption APIs so the keys can't be
even under debug.
If you're trying to take a zoned decimal number, encrypt it, and
store
in the database, you need to store the result in:
- A CHAR or VARCHAR column with the FOR BIT DATA attribute
the moment).hexwhich uses CCSID 65535.types
- A BINARY column
- A VARBINARY column
- A BLOB column
- Or the DDS file defined equivalent of one of the above.
The encrypted data is a binary string that requires one of those
data
to store the results. Encrypted data doesn't conform to the set of
Kentvalues that are allowable for storage in a zoned or packedcolumn,
decimal
Get the IBM Redbook "IBM System i Security: Protecting i5/OStalk
Data with Encryption". In the July 2008 version of that book,
see chapter 7 "Database Considerations", section 7.2, pages 78
and 79 in particular,
about storage requirements of the encrypted data.
Another great PDF book is "Protecting IBM i data with encryption"
by
forMilligan and Beth Hagemeister (March 2014). Read pages 31
through
33
COLUMN),storage requirements.COLUMN),
When you apply the field procedure to a column (ALTER TABLE
ALTER
it calls the field procedure to encrypt that column for ALL rows
of the table.
When you drop a field procedure from a column (ALTER TABLE ALTER
it calls the field procedure to decrypt that column for ALL rows
of the table.
I imagine a field procedure could be used for a non-encryption
purpose, although I've not tried that (no use case comes to mind
at
law.vhamberg@xxxxxxxxxxxxxxxAssuming that is allowed, a field procedure that returns zoneddata
decimal
for storage in a zoned decimal column is fine to do, but notencrypted
with
data.
HTH,
Mike
On Mon, Mar 23, 2020 at 4:48 PM Vernon Hamberg <
problems.wrote:
Nathan
Is the numeric column in question a key of the file? Are you
using native record-level access? If either, then RPG might
give you
I'vedon'tI thought it was only with keyed columns, but maybe not.
You might try using SQL to process the file.
Vern
On 3/23/2020 3:26 PM, Nathan Hughes wrote:
First I want to apologize for my ignorance in RPGLE...we
typically
use this programming language, but have to for FieldProc purposes.
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fresolution.researched this issue for a couple weeks now, and cannot find a
theI'm currently trying to write a field procedure program. I have
taken
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2IBM RPGLE example (
Fw
ww.ibm.com%2Fsupport%2Fknowledg...yfpexample.htm&data=01%7C01
%7
Cnathan.hughes%40badgepass.com%7Cb53cc8c797b842c7e97508d7d44ad949
%7
C224758f072634e52a6e4f5db2afdf99e%7C0&sdata=F6fRwGRLwT2%2F7nG
3o KYMocGBkMsP38fWksBeCDfuBj4%3D&reserved=0<
ww
w.ibm.com%2Fsupport%2Fknowledgecenter%2Fssw_ibm_i_73%2Fsqlp%2Frbaf
yf
pexample.htm&data=01%7C01%7Cnathan.hughes%40badgepass.com%7Cb5
3c
c8c797b842c7e97508d7d44ad949%7C224758f072634e52a6e4f5db2afdf99e%7C
0&
amp;sdata=5ZomdBu5GujxBVXT4u2PG10YG0z3hTUlReUgtUowCf0%3D&reser
ve
d=0
begin,),
and started to modify it to work for us.
Here is the table that was created for testing purposes:varchar, char, blob, etc., but I needed to add zoned numeric.
********** Beginning of data *************************************
A R REC
A ECNUM 16S 0 COLHDG('NUMBER')
A ALIAS(EC_NUMBER)
A ECALPHA 32A COLHDG('ALPHA')
A ALIAS(EC_ALPHA)
************* End of data
****************************************
The program from the link above allows for several SQL types,
clob,
To
Ion
addingstudied and stepped through the program to see what it was
doing, and thought I had a good understanding of how it worked.
I then started
what appeared as necessary for the SQL_TYP_ZONED (488).
When I send the 'ALTER TABLE' command and set the FieldProc
program
abut
column, the first call is to Register(function code: 8), the
next is Encode(function code: 0) to encode the values in said column.
I set a breakpoint at the end of the program, and evaluatedparameters when setting the FieldProc on a VARCHAR and ZONED
the
NUMERIC columns, and besides being different data types all
appears correct,
invalidthe ZONED NUMERIC column abruptly stops with the error below
after the register call.
Message ID . . . . . . : SQL0685 Date sent . . . . . . :
03/18/20 Time sent . . . . . . : 16:17:04
Message . . . . : Field procedure on column ECNUM has returned
invaliddata.
Cause . . . . . : Field procedure on column ECNUM has returned
attachments,data.
Recovery . . . : Change the field procedure to return valid data.
Thank you in advance for any help you can give on this.
Thanks,
Nathan Hughes
Software Developer
[BadgePass]
601.499.2131 Office
280 Trace Colony Park
Ridgeland, MS 39157
CONFIDENTIALITY NOTICE: This e-mail message, including any
is for the sole use of the intended recipient(s) and may
contain confidential and privileged information or otherwise
protected by
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2FarchAny
youunauthorized review, use, disclosure or distribution is
prohibited. If
are not the intended recipient, please contact the sender by
reply
and destroy all copies of the original message.
--
This is the RPG programming on IBM i (RPG400-L) mailing list To
post a message email: RPG400-L@xxxxxxxxxxxxxxxxxx To subscribe,
unsubscribe, or change list options,
visit:
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F
%2
Flists.midrange.com%2Fmailman%2Flistinfo%2Frpg400-l&data=01
%7
C01%7Cnathan.hughes%40badgepass.com%7Cb53cc8c797b842c7e97508d7d
44
ad949%7C224758f072634e52a6e4f5db2afdf99e%7C0&sdata=fR6WyyLi
gA
yk2oumpUr%2BY7MHUIuMusn%2FJgGqQP4pMV8%3D&reserved=0
or email: RPG400-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives at
ive.midrange.com%2Frpg400-l&data=01%7C01%7Cnathan.hughes%40badgepa
ss.com%7C2f041caafaef4ce7b10d08d7d4b7df81%7C224758f072634e52a6e4f5db2a
fdf99e%7C0&sdata=DYCnMwgpnadSuNGUGX62iqsTVSeVLmShKA6sHJzAjbU%3D&am
p;reserved=0
.
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Farchaffiliate
Please contact support@xxxxxxxxxxxx for any subscription
related questions.
Help support midrange.com by shopping at amazon.com with our
--link:
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F
%2
Famazon.midrange.com%2F&data=01%7C01%7Cnathan.hughes%40badg
ep
ass.com%7Cb53cc8c797b842c7e97508d7d44ad949%7C224758f072634e52a6
e4
f5db2afdf99e%7C0&sdata=lEGPfNdWfKJyzPyzZFvGzRgNwZeuOkmUEriq
ze
Dr5V8%3D&reserved=0
This is the RPG programming on IBM i (RPG400-L) mailing list To
post a message email: RPG400-L@xxxxxxxxxxxxxxxxxx To subscribe,
unsubscribe, or change list options,
visit:
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2
Fl
ists.midrange.com%2Fmailman%2Flistinfo%2Frpg400-l&data=01%7C0
1%
7Cnathan.hughes%40badgepass.com%7Cb53cc8c797b842c7e97508d7d44ad94
9%
7C224758f072634e52a6e4f5db2afdf99e%7C0&sdata=fR6WyyLigAyk2oum
pU
r%2BY7MHUIuMusn%2FJgGqQP4pMV8%3D&reserved=0
or email: RPG400-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives at
ive.midrange.com%2Frpg400-l&data=01%7C01%7Cnathan.hughes%40badgepa
ss.com%7C2f041caafaef4ce7b10d08d7d4b7df81%7C224758f072634e52a6e4f5db2a
fdf99e%7C0&sdata=DYCnMwgpnadSuNGUGX62iqsTVSeVLmShKA6sHJzAjbU%3D&am
p;reserved=0
.
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Farch--
Please contact support@xxxxxxxxxxxx for any subscription related
questions.
Help support midrange.com by shopping at amazon.com with our
affiliate
link:
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2
Fa
mazon.midrange.com%2F&data=01%7C01%7Cnathan.hughes%40badgepass.
com%7Cb53cc8c797b842c7e97508d7d44ad949%7C224758f072634e52a6e4f5db
2a
fdf99e%7C0&sdata=lEGPfNdWfKJyzPyzZFvGzRgNwZeuOkmUEriqzeDr5V8%
3D
&reserved=0
This is the RPG programming on IBM i (RPG400-L) mailing list To
post a message email: RPG400-L@xxxxxxxxxxxxxxxxxx To subscribe,
unsubscribe, or change list options,
visit:
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2F
li
sts.midrange.com%2Fmailman%2Flistinfo%2Frpg400-l&data=01%7C01%
7C
nathan.hughes%40badgepass.com%7Cb53cc8c797b842c7e97508d7d44ad949%7
C2
24758f072634e52a6e4f5db2afdf99e%7C0&sdata=fR6WyyLigAyk2oumpUr%
2B
Y7MHUIuMusn%2FJgGqQP4pMV8%3D&reserved=0
or email: RPG400-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives at
ive.midrange.com%2Frpg400-l&data=01%7C01%7Cnathan.hughes%40badgepa
ss.com%7C2f041caafaef4ce7b10d08d7d4b7df81%7C224758f072634e52a6e4f5db2a
fdf99e%7C0&sdata=DYCnMwgpnadSuNGUGX62iqsTVSeVLmShKA6sHJzAjbU%3D&am
p;reserved=0
.
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Farch
Please contact support@xxxxxxxxxxxx for any subscription related
questions.
Help support midrange.com by shopping at amazon.com with our
affiliate
link:
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2F
am
azon.midrange.com%2F&data=01%7C01%7Cnathan.hughes%40badgepass.
co
m%7Cb53cc8c797b842c7e97508d7d44ad949%7C224758f072634e52a6e4f5db2af
df
99e%7C0&sdata=lEGPfNdWfKJyzPyzZFvGzRgNwZeuOkmUEriqzeDr5V8%3D&a
mp
;reserved=0
--
This is the RPG programming on IBM i (RPG400-L) mailing list To post
a message email: RPG400-L@xxxxxxxxxxxxxxxxxx To subscribe,
unsubscribe, or change list options,
visit:
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fli
st
s.midrange.com%2Fmailman%2Flistinfo%2Frpg400-l&data=01%7C01%7Cna
th
an.hughes%40badgepass.com%7Cb53cc8c797b842c7e97508d7d44ad949%7C22475
8f
072634e52a6e4f5db2afdf99e%7C0&sdata=fR6WyyLigAyk2oumpUr%2BY7MHUI
uM
usn%2FJgGqQP4pMV8%3D&reserved=0
or email: RPG400-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives at
ive.midrange.com%2Frpg400-l&data=01%7C01%7Cnathan.hughes%40badgepa
ss.com%7C2f041caafaef4ce7b10d08d7d4b7df81%7C224758f072634e52a6e4f5db2a
fdf99e%7C0&sdata=DYCnMwgpnadSuNGUGX62iqsTVSeVLmShKA6sHJzAjbU%3D&am
p;reserved=0
.
questions.
Please contact support@xxxxxxxxxxxx for any subscription related
--
Help support midrange.com by shopping at amazon.com with our
affiliate
link:
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fam
az
on.midrange.com%2F&data=01%7C01%7Cnathan.hughes%40badgepass.com%
7C
b53cc8c797b842c7e97508d7d44ad949%7C224758f072634e52a6e4f5db2afdf99e%
7C
0&sdata=lEGPfNdWfKJyzPyzZFvGzRgNwZeuOkmUEriqzeDr5V8%3D&reser
ve
d=0
This is the RPG programming on IBM i (RPG400-L) mailing list To post a
message email: RPG400-L@xxxxxxxxxxxxxxxxxx To subscribe, unsubscribe,
or change list options,
visit:
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flist
s.midrange.com%2Fmailman%2Flistinfo%2Frpg400-l&data=01%7C01%7Cnath
an.hughes%40badgepass.com%7C2f041caafaef4ce7b10d08d7d4b7df81%7C224758f
072634e52a6e4f5db2afdf99e%7C0&sdata=CS%2F5qpyd0wluftxQhRW6Ftje%2BA
XH7rteVQGiHU6BeWI%3D&reserved=0
or email: RPG400-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives at
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Farch
ive.midrange.com%2Frpg400-l&data=01%7C01%7Cnathan.hughes%40badgepa
ss.com%7C2f041caafaef4ce7b10d08d7d4b7df81%7C224758f072634e52a6e4f5db2a
fdf99e%7C0&sdata=DYCnMwgpnadSuNGUGX62iqsTVSeVLmShKA6sHJzAjbU%3D&am
p;reserved=0
.
Please contact support@xxxxxxxxxxxx for any subscription related
questions.
Help support midrange.com by shopping at amazon.com with our affiliate
link:
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Famaz
on.midrange.com%2F&data=01%7C01%7Cnathan.hughes%40badgepass.com%7C
2f041caafaef4ce7b10d08d7d4b7df81%7C224758f072634e52a6e4f5db2afdf99e%7C
0&sdata=7NqHhBWUrbgfYaXMQuzeEoU7fRblqyV9UInjUl4KmDE%3D&reserve
d=0 CONFIDENTIALITY NOTICE: This e-mail message, including any
attachments, is for the sole use of the intended recipient(s) and may
contain confidential and privileged information or otherwise protected
by law. Any unauthorized review, use, disclosure or distribution is
prohibited. If you are not the intended recipient, please contact the
sender by reply e-mail and destroy all copies of the original message.
--
This is the RPG programming on IBM i (RPG400-L) mailing list To post a
message email: RPG400-L@xxxxxxxxxxxxxxxxxx To subscribe, unsubscribe,
or change list options,
visit:
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flist
s.midrange.com%2Fmailman%2Flistinfo%2Frpg400-l&data=01%7C01%7Cnath
an.hughes%40badgepass.com%7C2f041caafaef4ce7b10d08d7d4b7df81%7C224758f
072634e52a6e4f5db2afdf99e%7C0&sdata=CS%2F5qpyd0wluftxQhRW6Ftje%2BA
XH7rteVQGiHU6BeWI%3D&reserved=0
or email: RPG400-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives at
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Farchive.midrange.com%2Frpg400-l&data=01%7C01%7Cnathan.hughes%40badgepass.com%7C2f041caafaef4ce7b10d08d7d4b7df81%7C224758f072634e52a6e4f5db2afdf99e%7C0&sdata=DYCnMwgpnadSuNGUGX62iqsTVSeVLmShKA6sHJzAjbU%3D&reserved=0.
Please contact support@xxxxxxxxxxxx for any subscription related
questions.
Help support midrange.com by shopping at amazon.com with our affiliate
link:
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Famaz
on.midrange.com&data=01%7C01%7Cnathan.hughes%40badgepass.com%7C2f0
41caafaef4ce7b10d08d7d4b7df81%7C224758f072634e52a6e4f5db2afdf99e%7C0&a
mp;sdata=mAEoOTxJnZPj34xFSEkFGYfN%2BMkfPjrkCikd0fDZ0a4%3D&reserved
=0
As an Amazon Associate we earn from qualifying purchases.
This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].
Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.