Re: Content-type issue with application/json and application/javascript -- WEB400

gents

i have just uploaded the latest version of powerext core (5.02) that
includes
yajl and a httpapi example that calls a powerext web service.

the program (yajlreq/yajl) are in the example library and can be used for
skeletons for testing.

On Wed, Aug 26, 2015 at 7:28 PM, Matt Lavinder <
mlavinder@xxxxxxxxxxxxxxxxxxx> wrote:

Thanks for the suggestions but I managed to modify to CGIDEV2 to do what I
need. I'll have to do some more testing, but I might send it over to
Giovanni and see if he can add my procedure to a future release.

I didn't screw around with the base code. I wound up adding a procedure to
re-encode the body just before it is sent to the client. It sort of
mirrors the WrtHTMLToStmf that is already included.

You pass in the HTTP headers and the target CCSID. The procedure spits out
the HTTP headers and re-encodes everything in the buffer to the CCSID that
was provided. Seems to work well. Probably not the most efficient way of
doing it, but it allows me to solve my issue without messing with any of
the existing procedures in CGIDEV2. I have no issue sacrificing a couple
milliseconds of performance to keep things stable.

Thanks for all the tips and examples.

On Wed, Aug 26, 2015 at 12:38 PM, Bradley Stone <bvstone@xxxxxxxxx> wrote:

I don't take it as nasty, but it's also nothing to do with what this was
doing or it's purpose. The JSON file is static in this case (but, it
certainly could be dynamic and created with yajl or anything else).

The OP had problems using the application/json content-type header and

the

output not getting "converted".

This is an example of how we updated our eRPG SDK toolkit to handle this
problem.

Brad
www.bvstools.com

On Wed, Aug 26, 2015 at 11:33 AM, Henrik Rützou <hr@xxxxxxxxxxxx> wrote:

bradley

i'm not trying to be 'nasty' but i'm converting from rpg based json

support

by including yajl c support in powerext - yajl work 4-5 times faster
creating
json than native rpg.

On Wed, Aug 26, 2015 at 6:29 PM, Henrik Rützou <hr@xxxxxxxxxxxx>

wrote:

bradley

where do the mime header come from?

how do you process the incomming json?

how do you handle different clients, one may send you text/plain and
another
application/json and expext to get the same content-type back?

On Wed, Aug 26, 2015 at 6:11 PM, Bradley Stone <bvstone@xxxxxxxxx>

wrote:

Well, I've updated our eRPG SDK to allow you to set the CCSID of the
output
for situations like this... it was a pretty easy fix.

The example is here:
http://erpgsdk.bvstools.com/demo/

Look down the right for the DSPJSON example and you can run the

program

as

well as view the template file and JSON file used (which was

retrieved

from
Google Geocoding API for an address of the white house).

Brad
www.bvstools.com

On Tue, Aug 25, 2015 at 11:00 PM, Henrik Rützou <hr@xxxxxxxxxxxx>

wrote:

bradley

i completely have forgotten that little trick ;-) as far as i

remember

ms-office also works with it but i wouldn't rely on the method

today

and not with json/webservices.

On Wed, Aug 26, 2015 at 5:00 AM, Bradley Stone <bvstone@xxxxxxxxx

wrote:

I should state this was just in Chrome and I understand most

clients

probably will ignore the Content-encoding header... but it was a

fun

find.

:)

Brad
www.bvstools.com

On Tue, Aug 25, 2015 at 5:09 PM, Bradley Stone <

bvstone@xxxxxxxxx

wrote:

Henrik,

I found something you posted a few years ago and tried it and

it

worked

for me...

Content-type: application/json
Content-encoding: EBCDIC

Is this something deemed incorrect or not a solution? I was

actually

surprised it worked. :)

Maybe there is a better solution? It seems we would need to

convert

all

the output to a specific CCSID before writing it out to StdOut

otherwise

(not a huge issue, but may be for some that can't just modify

their

toolkits to do this).

Brad
www.bvstools.com

On Tue, Aug 25, 2015 at 3:41 PM, Matt Lavinder <
mlavinder@xxxxxxxxxxxxxxxxxxx> wrote:

Henrik -

Already downloaded it. :)

I noticed you example HTTP instance that you make UTF-8 the

default

encoding. I can't do that. If possible, could you send me a

very

simple

example of how to make this work in PowerEXT Core without

changing

the

HTTP
Server configuration? I see the procedures for turning on

UTF-8

mode

but

didn't see any examples that use them. Plus, you said the

headers

cannot

be UTF-8 encoded.

I already have a CGIDEV2 template setup for my JSON, so I

really

don't

want
to change gears and use the JSON library, but I have no issue

doing a

find

replace and changing WRTSECTION to something else and using

the

PowerEXT

Core. I don't even mind changing the section and variable

tags

if

I

need

to.

On Tue, Aug 25, 2015 at 2:18 PM, Henrik Rützou <

hr@xxxxxxxxxxxx>

wrote:

mark

if you work a lot within this area you will find that ibm i

installations

seldom is able to dictate the content-type

text/json isn't btw an official mime type

On Tue, Aug 25, 2015 at 7:50 PM, Mark Murphy/STAR BASE

Consulting

Inc. <

mmurphy@xxxxxxxxxxxxxxx> wrote:

Just out of pure ignorance, since application/json causes

conversion

issues, and text/json appears to work properly, why do

you

want to

use

application/json? Are there issues that text/json causes

on

the

client

side?

Mark Murphy
STAR BASE Consulting, Inc.
mmurphy@xxxxxxxxxxxxxxx

-----Matt Lavinder <mlavinder@xxxxxxxxxxxxxxxxxxx>

wrote:

-----

To: "Web Enabling the IBM i (AS/400 and iSeries)" <

web400@xxxxxxxxxxxx>

From: Matt Lavinder <mlavinder@xxxxxxxxxxxxxxxxxxx>
Date: 08/25/2015 11:42AM
Subject: Re: [WEB400] Content-type issue with

application/json

and

application/javascript

Henrik -

No worries. I have no intention of changing our

CGIConvMode,

but

I

did

want to make sure I understood the implications. The

bulk

of

our

CGI

programs produce text/html and they work perfectly fine.

I plan to tweak the programs that produce JSON output and

leave

everything

else alone.

On Tue, Aug 25, 2015 at 11:36 AM, Henrik Rützou <

hr@xxxxxxxxxxxx>

wrote:

btw if you use binary or recieve application/xxxx

content

type

the

input

also
is recieved as binary without conversion to ebcdic

On Tue, Aug 25, 2015 at 5:34 PM, Henrik Rützou <

hr@xxxxxxxxxxxx

wrote:

matt

you are right

On Tue, Aug 25, 2015 at 5:31 PM, Matt Lavinder <
mlavinder@xxxxxxxxxxxxxxxxxxx> wrote:

Kevin -

It's OK because I know enough to get what he was

saying.

It

makes

it

pretty clear in the HTTP Server documentation that

if

you

add

"charset=utf-8"
(really, charset=anything) that no conversion

occurs

on

the

body.

I have never tried CGIConvMode %BINARY%, but

wouldn't

that

mean

that

all

your content has to be UTF-8, including output with

"content-type:text/*"?

On Tue, Aug 25, 2015 at 11:23 AM, Kevin Turner <
kevin.turner@xxxxxxxxxxxxxxxxxxxx> wrote:

Henrick

I know English is not your first language, but I

couldn't

understand

that

at all :)

But anyway, my observations are not relevant

because

we

always

use

"CGIConvMode BINARY"

Kevin

-----Original Message-----
From: WEB400 [mailto:web400-bounces@xxxxxxxxxxxx]

On

Behalf

Of

Henrik

Rützou
Sent: 25 August 2015 16:19
To: Web Enabling the IBM i (AS/400 and iSeries) <

web400@xxxxxxxxxxxx>

Subject: Re: [WEB400] Content-type issue with

application/json

and

application/javascript

apache doesn't neither converts from an ebcdic

body

if

you

send

then

the

body also has to be pre-encoded in utf-8

Content-Type: text/html; charset=utf-8

On Tue, Aug 25, 2015 at 5:14 PM, Kevin Turner <
kevin.turner@xxxxxxxxxxxxxxxxxxxx> wrote:

I am pretty sure we always ensure the data is in

UTF-8

when

sending

content that has a content type that is anything

but

"text/...."

-----Original Message-----
From: WEB400 [mailto:

web400-bounces@xxxxxxxxxxxx]

On

Behalf

Of

Matt

Lavinder
Sent: 25 August 2015 16:12
To: Web Enabling the IBM i (AS/400 and iSeries)

<

web400@xxxxxxxxxxxx>

Subject: Re: [WEB400] Content-type issue with

application/json

and

application/javascript

Hello Henrick. That's sort of what I suspected.

Do

I

have

to

change

the

CGI program to produce ASCII output, or is

there a

way to

tell

Apache

to do an EBCDIC to ASCII conversion on an

"application/javascript"

content type?

On Tue, Aug 25, 2015 at 10:56 AM, Henrik Rützou

<

hr@xxxxxxxxxxxx>

wrote:

apache converts ebcdic to utf-8 if the content

type

is

text/...

othervise not.

On Tue, Aug 25, 2015 at 4:52 PM, Matt

Lavinder <

mlavinder@xxxxxxxxxxxxxxxxxxx> wrote:

We have several CGI programs that generate

javascript,

usually

JSON.

For years we have set the content type to

text/plain

or

text/javascript. I'd like to start using

the

more

standard

"application/json", but I am

hitting

a major snag. The output is garbage. I get

the

exact

same

output

when I set the content-type to

application/javascript.

Just to verify this was related to CGI

output, I

used

AddType

to

change

the

content-type of static JavaScript files to

"application/javascript".

Those

files look fine, so this must have something

to

do

with

it

being

CGI output.

I don't really know what is up with the

output,

but I

know

it

isn't EBCDIC. It doesn't convert. Garbage

in,

garbage

out.

Changing the

content

type back to text/javascript sets everything

right.

Anyone have any guesses what is going on

here?

We

use

CGIDEV2

under IBM i 7.1. The default CCSID of our

CGI

jobs

is

37.

Just so we are clear on one thing. We had

the

following

settings

in our HTTP Conf:

DefaultFsCCSID 037
CGIConvMode %%EBCDIC/EBCDIC%%

I commented both settings out and it had no

impact on

the

output

whatsoever. All of our CGI programs that

use

content

type

of

"text/*"

seemed to work. The ones using

"application/javascript"

or

"application/json" for content type still

looked

like

garbage.

The only guess I have is that mod_cgi sees

that

the

content-type

starts with "application" and treats the

output

as

binary,

but I

have no idea how I

can

tell mod_cgi to treat

"application/javascript"

as

text,

or

if

that

is

even

the issue.

I am hoping someone has an idea. I know

Apache

pretty

well,

but

this one has me stumped.

--
Matt
--
This is the Web Enabling the IBM i (AS/400

and

iSeries)

(WEB400)

mailing list To post a message email:

WEB400@xxxxxxxxxxxx

To

subscribe, unsubscribe, or change list

options,

visit:

http://lists.midrange.com/mailman/listinfo/web400

or email: WEB400-request@xxxxxxxxxxxx

Before

posting,

please

take

a moment to review the archives at

http://archive.midrange.com/web400.

--
Regards,
Henrik Rützou

http://powerEXT.com <http://powerext.com/>
--
This is the Web Enabling the IBM i (AS/400 and

iSeries)

(WEB400)

mailing list To post a message email:

WEB400@xxxxxxxxxxxx

To

subscribe, unsubscribe, or change list

options,

visit:

http://lists.midrange.com/mailman/listinfo/web400

or email: WEB400-request@xxxxxxxxxxxx Before

posting,

please

take a

moment to review the archives at

http://archive.midrange.com/web400

.

--

*Matt Lavinder Programmer AnalystData Management

Inc.Phone:

(336)

573-5045Fax: (336) 573-5001*
--
This is the Web Enabling the IBM i (AS/400 and

iSeries)

(WEB400)

mailing list To post a message email:

WEB400@xxxxxxxxxxxx

To

subscribe, unsubscribe, or change list options,
visit:

http://lists.midrange.com/mailman/listinfo/web400

or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review

the

archives

at

http://archive.midrange.com/web400.

___________________________________________
This email has been scanned by iomartcloud.
http://www.iomartcloud.com/

________________________________

NOTICE: The information in this electronic mail

transmission

is

intended by CoralTree Systems Ltd for the use of

the

named

individuals

or entity to which it is directed and may

contain

information

that

is

privileged or otherwise confidential. If you

have

received

this

electronic mail transmission in error, please

delete

it

from

your

system without copying or forwarding it, and

notify

the

sender

of

the

error by reply email or by telephone, so that

the

sender's

address

records can be corrected.

----------------------------------------------------------------------

----------

CoralTree Systems Limited
Company Registration Number 5021022.
Registered Office:
12-14 Carlton Place
Southampton
Hampshire
SO15 2EA
VAT Registration Number 834 1020 74.
--
This is the Web Enabling the IBM i (AS/400 and

iSeries)

(WEB400)

mailing list To post a message email:

WEB400@xxxxxxxxxxxx

To

subscribe, unsubscribe, or change list options,
visit:

http://lists.midrange.com/mailman/listinfo/web400

or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review

the

archives

at

http://archive.midrange.com/web400.

--
Regards,
Henrik Rützou

http://powerEXT.com <http://powerext.com/>
--
This is the Web Enabling the IBM i (AS/400 and

iSeries)

(WEB400)

mailing

list To post a message email: WEB400@xxxxxxxxxxxx

To

subscribe,

unsubscribe, or change list options,
visit:

http://lists.midrange.com/mailman/listinfo/web400

or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the

archives

at

http://archive.midrange.com/web400.

___________________________________________
This email has been scanned by iomartcloud.
http://www.iomartcloud.com/

________________________________

NOTICE: The information in this electronic mail

transmission

is

intended

by CoralTree Systems Ltd for the use of the named

individuals

or

entity

to

which it is directed and may contain information

that

is

privileged

or

otherwise confidential. If you have received this

electronic

mail

transmission in error, please delete it from your

system

without

copying or

forwarding it, and notify the sender of the error

by

reply

email

or

by

telephone, so that the sender's address records

can

be

corrected.

--------------------------------------------------------------------------------

CoralTree Systems Limited
Company Registration Number 5021022.
Registered Office:
12-14 Carlton Place
Southampton
Hampshire
SO15 2EA
VAT Registration Number 834 1020 74.
--
This is the Web Enabling the IBM i (AS/400 and

iSeries)

(WEB400)

mailing

list
To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit:

http://lists.midrange.com/mailman/listinfo/web400

or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the

archives

at http://archive.midrange.com/web400.

--

*Matt Lavinder Programmer AnalystData Management

Inc.Phone:

(336)

573-5045Fax: (336) 573-5001*
--
This is the Web Enabling the IBM i (AS/400 and

iSeries)

(WEB400)

mailing

list
To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit:

http://lists.midrange.com/mailman/listinfo/web400

or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the

archives

at http://archive.midrange.com/web400.

--
Regards,
Henrik Rützou

http://powerEXT.com <http://powerext.com/>

--
Regards,
Henrik Rützou

http://powerEXT.com <http://powerext.com/>
--
This is the Web Enabling the IBM i (AS/400 and iSeries)

(WEB400)

mailing

list
To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit:

http://lists.midrange.com/mailman/listinfo/web400

or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the

archives

at http://archive.midrange.com/web400.

--

*Matt Lavinder Programmer AnalystData Management

Inc.Phone:

(336)

573-5045Fax: (336) 573-5001*
--
This is the Web Enabling the IBM i (AS/400 and iSeries)

(WEB400)

mailing

list
To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the

archives

at http://archive.midrange.com/web400.
--
This is the Web Enabling the IBM i (AS/400 and iSeries)

(WEB400)

mailing

list
To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the

archives

at http://archive.midrange.com/web400.

--
Regards,
Henrik Rützou

http://powerEXT.com <http://powerext.com/>
--
This is the Web Enabling the IBM i (AS/400 and iSeries)

(WEB400)

mailing

list
To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/web400.

--

*Matt Lavinder Programmer AnalystData Management Inc.Phone:

(336)

573-5045Fax: (336) 573-5001*
--
This is the Web Enabling the IBM i (AS/400 and iSeries)

(WEB400)

mailing

list
To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/web400.

--
This is the Web Enabling the IBM i (AS/400 and iSeries) (WEB400)

mailing

list
To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/web400.

--
Regards,
Henrik Rützou

http://powerEXT.com <http://powerext.com/>
--
This is the Web Enabling the IBM i (AS/400 and iSeries) (WEB400)

mailing

list
To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/web400.

--
This is the Web Enabling the IBM i (AS/400 and iSeries) (WEB400)

mailing

list
To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/web400.

--
Regards,
Henrik Rützou

http://powerEXT.com <http://powerext.com/>

--
Regards,
Henrik Rützou

http://powerEXT.com <http://powerext.com/>
--
This is the Web Enabling the IBM i (AS/400 and iSeries) (WEB400)

mailing

list
To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/web400.

--
This is the Web Enabling the IBM i (AS/400 and iSeries) (WEB400) mailing
list
To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/web400.

--

*Matt Lavinder Programmer AnalystData Management Inc.Phone: (336)
573-5045Fax: (336) 573-5001*
--
This is the Web Enabling the IBM i (AS/400 and iSeries) (WEB400) mailing
list
To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/web400.