problem with coding characters file Json coming from App

[gio.cot]

I try to explain this problem of mine (even if it is not easy); my friends has developed an App for Android that exchanges data with As400 through of the calls of Web Service AS400 that send / read file Json (in practice some WSs serve for importing data on APP therefore they send data JSon to the Tablet, while other WSs receive data JSon from the Tablet). We have implemented these WSs starting from the examples furnished in the site http://www.powerext.com/2015/06/03/my-first-wp-homepage /; everything has regularly worked up to the day before yesterday, when suddenly the call to the WS, from one of the two devices, that had to read and to unload the data on As400 has begun to go to error, as the function "storeToStmf(jsonStore: '/ myjson.json':1208) " (you see link above according to example it rules 86) was recalled passing him -1 in the parameter jsonStore. From what I have understood, it seems that the value -1 have passed when the file Json is empty (I have made some verifications and the file Json Non it is empty) or it has some other problems of encoding or something similar. Premised that the other device keeps on regularly working and the version of the App is the same one on both the devices and on As400 y has not been modified anything, I don't absolutely succeed in understanding from thing can spring this problem; the only things that can come me to mind are:
1. could have been change some setings in the tablet that has influenced in the use of the coding of the characters of the file Json, that is produced by the device?
2. an updating of the version of Android could have been performed that is causing this problem? (almost surely Android has been adjourned the same day when they have begun out the problems. N.B. in the device in which I have problems I have Android 5.1.1, while in the other one I have the 4.3)
To try to understand what the problem could be, I have made a program ex new that goes to read the data Json sent by the Device (following the suggestion of Henrik Rützou owner http://powerEXT.com) and it performs how much under brought:

//?Clear powerEXT Service Program & Response Objects
clearSrvPgm();
//?get input from Apache
httpStore = getExtInputRaw();
// Process the request
// .. save what is received for testing
storeToStmf(httpStore: '/tmp//myjson.json'+ %Trim(%Char(%TimeStamp)):1208);
the content of the file coming from the device "incriminated", and saved in the IFS it is the following:




************Inizio dei dati****************
BtBb'CaC»ChCaBØ# CdCý^Ca ChCjBbBª BtBbCu;CuBµ CuC~ Bµ' BbBªBbBbBðBb'C£ ;# CuBØC¿ BØ CjCdCd
BbBðBbBµBµ;C¡C£Ct;C£ ';'Cs BbBªB°BðBbC¡C£Ct;C£ ';'Cs BbBªB°BðBb CÝ ;C¡C£Cv*
BØCt Cd Ce BbBðBbBµBµ;C¡C£Ct;C£ ';'Cs BbBªB°BðBbC¡C£Ct;C£ ';'Cs BbBªB°BðB
BbBªB°BðBb C~C¡Cu;C¿Cu BbBªB°BðBb C~C¡Cu;C¿CuBqBbBªB°BðBbCu #; BµCvBµCzCzC£ B
Cu;C¿CuBlBbBªB°BðBb C~C¡Cu;C¿CuBmBbBªB°BðBb C~C¡Cu;C¿CuBnBbBªB°BðBb C~C¡Cu;C¿C
************Fine dati**********************

While the content of the file coming from the device Ok, and saved in the Ifs, it is the following:
************Inizio dei dati****************
£ }ÁËÈÁ {?ÄÍ_Á>ÈÑ $£ { } ^ä!+ëáå+ < }ñ}í{ñ+á ñ(&!+^
ñ(&!+^ ñ(&!+^ ä!{^ä íë^( å éé îá ä!{^î <í}
ä!{^ä<ñá+}á^ã } ^ ää!+}!^ë^ë <{! ä!
!ê ^ë} (& ë^ñ+îñ }!^ {^ ë + !ê ^ñ+ëáêñ(
+!}á ä!{^( å ééñ+! !ê ^ñ+îñ!^ {^ ë ä!{
êñã^ñ+}^}â!< +í(^êñå\á^{!ä í}á+}á <ñ]í^ñî ^
<ñ]í^ñî ^ ä^ä +äá<< }! }ñ&!^ñî ^ { } ^( +í}á
}ñ&!^ñî ^ <ñ]í^ñî ^ +í(^{!ä }ñ&!^ñî ^
<ñ]í^ñî ^ }ñ&!^ñî ^ ä!{^é!+ ^ ((ñ+ ä!{^ä<ñá
}!}^ñ+ä ëë }! ^êñã^ñ+}^}â!< êñã^!ê{ñ+á !ê ^{!ä
}}!^ñ(&!+ { } ^ñ+îñ!^ {^ ë { } ^ë} (&
}ñ+! ñî ^ ñî ^ ñî ^ ñî ^ ñî ^
ñ+!^ ^î î { } ^{!ä ñ(&!+^!( ååñ { } ^êáåñ
ë^{!ä^ ä]íñëñ}! + ë^{!ä^ë} (& }! ë ++!^{!ä
!+^ ñ(&!+^ ä!{^ëä!+}! ä!{^(!{^}ê ë& ') êÑÅÇ
<ñá+}á^ë&á { } ^ä!+ëáå+ ä!{^ ê}ñä!<!
^ñî }ñ&!^{!ä ã åêí&&!^ä<ñá+}á { } ^ñ+îñ!^ {^ ë
&áê^ <ñ]í^ñî ñ(&!ê}!^+á}}!^êñå ä!{^( êä\ñ!
************Fine dati**********************
Now beyond the fact that both the documents are incomprehensible (I have to understand well thing they make these procedures that I have used in the program) the other strange thing is that if from the EDTF I perform F15 in the first file (the incriminated one) I see:

1. Copia da file flusso . . . . /tmp/myjson.json2015-11-05-15.34.45.493000
2. Copia da file database . . . . Nome
Libreria. . . . . . . . . . . Nome, *LIBL, *CURL
Membro . . . . . . . . . . . Nome, *FIRST
3. Modifica CCSID del file. . . . 00500 CCSID lav: 00280
4. Modifica CCSID della riga . . *NONE
5. Opzione EOL file di flusso . . *LF -- *CR, *LF, *CRLF, *LFCR, *USRDFN
Definito dall'utente . . . . Valore esadecimale

While in the other one I see:
1. Copia da file flusso . . . . /tmp/myjson.json2015-11-05-15.35.34.027000
2. Copia da file database . . . . Nome
Libreria. . . . . . . . . . . Nome, *LIBL, *CURL
Membro . . . . . . . . . . . Nome, *FIRST
3. Modifica CCSID del file. . . . 01208 CCSID lav: 00280
4. Modifica CCSID della riga . . *NONE
5. Opzione EOL file di flusso . . *USRDFN -- *CR, *LF, *CRLF, *LFCR, *USRDFN
Definito dall'utente . . . . Valore esadecimale

As it is possible that has taken two different CCSIDs and different EOL? if I have well understands the CCSID should take that brought in the call to the procedure "storeToStmf?.. :1208)

Don't I know thing to check to verify anymore.. there is someone that can give me some right-hand / suggestions?
Thanks in advance

[chris_hird]

Re: problem with coding characters file Json coming from App

Gio

From the information I can see you have a problem with the CCSID. The first one has CCSID of 0500 and the second 1208, yet you call the process to handle the file with CCSID 1208. That seems to be the problem here??

Chris..

[gio.cot]

Re: problem with coding characters file Json coming from App

I think to have found the problem !! if in the Json there are special character like ?ò? (in my Json I have a string ?Niccolò") it seems to generate problem , in fact now that I stored in the Device that no give me problem this character (ò), for generate a Json with inside it, I receive the same error like in the other device. What I have done: with debug I stopped in ? convCCSdbcs? procedure and after the call at ?iconv? (see below rc = iconv(ic:addr:size:toaddr:tosize), I have changed the ?rc? value to *zero and modified the value of ?tosize? to *zero; with this manipulation the Json file came saved in the Ifs as in the attached picture. you can notice that the ?ò? is translated in ?2? and from here on out there is not translation ..


Question : how can solve it in such way that ?iconv? works fine ?? or which workaround can use ?
Thanks

  addr = CcsAddr;                          
  size = CcsSize;                          
  toaddr = CcstoAddr;                      
  tosize = CcstoSize;                      
  from_CcsId = CcsFrom;                    
  to_CcsId = CcsTo;                        
  ic = QtqIconvOpen(to:from);              
  rc = iconv(ic:addr:size:toaddr:tosize);  
  iconv_close(ic);                         
  if rc > 0;                               
    return -1;                             
  endif;                                   
  return CcstoSize-tosize;

[Henrik Rutzou]

Re: problem with coding characters file Json coming from App

This is a problem on how to set up ICONV to skip or replace invalid characters that isn't represented
in the EBCDIC CCSid

Anyone know how to do that?