Re: Unzipping JSON files then parsing.

[VERNON HAMBERG Owner <vhamberg@xxxxxxxxxxxxxxx>]

Hi all


CPYFRMARCF/CPYTOARCF were introduced around 7.2 - they are pretty basic in their workings and do have some issues with library objects, where "files" are sometimes treated as directories - I think source PFs are like that. But I don't think these commands are meant to work with library objects, anyhow. Simon Hutchinson had an article on the use of these, google for CPYFRMARCF and you'll find that and others.


Regards
Vern


On Fri, 29 Mar, 2024 at 7:34 AM, Scott Klement <sk@xxxxxxxxxxxxxxxx> wrote:
 

To: rpg400-l@xxxxxxxxxxxxxxxxxx

Personally, I think it's probably a good assumption to assume that the 
extracted JSON files will always be UTF-8.  That's pretty much the 
standard for JSON. I know people (especially on IBM i) will sometimes 
store them in EBCDIC, but it's usually only temporary.  Stuff stored 
permanently or sent over a network is pretty much always UTF-8.

Assuming you agree...  just add some code to do CHGATR 
OBJ('/path/to/file.json') ATR(*CCSID) VALUE(1208) on each file before 
you read it in your program.  This forces the CCSID to 1208, so it 
should be interpreted correctly.

Background:  Storing the CCSID of a file in the filesystem is a 
unique-ish feature of IBM i. This concept is unknown to other platforms 
like Windows.  ZIP files, therefore, do not have a way to store the 
CCSID of the file.  they don't know what your CCSID should be!  It 
sounds like the CPYFRMARCF tool (I've not heard of this tool before) 
assumes that all files it extracts should be assigned the system ccsid 
(which for US systems is likely 37).  So you'll always get 37 instead of 
the actual format of the file.  As such, you will need to "know" the 
true CCSID and use CHGATR to set it properly.


On 3/28/24 4:05 PM, Greg Wilburn wrote:
> I'm having an issue parsing JSON files with YAJLINTO.
>
> I have an FTP Script that retrieves .ZIP files (containing the JSON files) from our FTP server.
> I then use CPYFRMARCF to extract the JSON files on the IFS.
>
>  From there, I am able to open them with Firefox or Notepad, but I cannot use WRKLNK to read them.
>
> The CCSID of the zip file is 819
> The CCSID of the extracted JSON files is 37
>
> I don't see how to control the CCSID of the JSON file(s) created by CPYFRMARCF (assuming this is my problem).
> If I extract on the PC, Notepad tells me the files are UTF-8.
>
> Any input would be appreciated.
> (I'm out until Monday)
>
> Thanks,
> Greg
>
> The actual error is:
>
> Message ID . . . . . . :   RNX0453       Severity . . . . . . . :   50
> Message type . . . . . :   Escape
> Date sent  . . . . . . :   03/28/24      Time sent  . . . . . . :   16:45:30
>
> Message . . . . :   An error occurred during conversion from CCSID(1208) to
>    CCSID(937).
> Cause . . . . . :   In RPG procedure GET_INPUT_ in program YAJL/YAJLINTO, an
>    error occurred during conversion of data between CCSID(1208) and CCSID(937)
> Recovery  . . . :   Check the joblog for previous messages, and contact the
>    person responsible for program maintenance to determine the cause of the
>    problem.
> [Logo]<https://www.totalbizfulfillment.com/><https://www.totalbizfulfillment.com/>>    Greg Wilburn
> Director of IT
> 301.895.3792 ext. 1231
> 301.895.3895 direct
> gwilburn@xxxxxxxxxxxxxxxxxxxxxxx<mailto:gwilburn@xxxxxxxxxxxxxxxxxxxxxxx><mailto:gwilburn@xxxxxxxxxxxxxxxxxxxxxxx<mailto:gwilburn@xxxxxxxxxxxxxxxxxxxxxxx>>
> 1 Corporate Dr
> Grantsville, MD 21536
> www.totalbizfulfillment.com<http://www.totalbizfulfillment.com><http://www.totalbizfulfillment.com<http://www.totalbizfulfillment.com>>