CommTCP_FD_SET error after HTTP_REQ

[c0nfitty]

We've been using HTTPAPI for quite a while now but we've never seen this error.
At seemingly random times the procedure COMMTCP_FD_SET, and sometimes COMMTCP_FD break on a substring operation.

Every time this happens it's trying to substring peFDSet starting at position 32. peFDSet is only 28 bytes.
The httpapi_debug logs basically show nothing. even with the debug level set at 2 this is the only thing in the debug logs...
It never gets to the "Set SNI Hostname" operation.

Any ideas?

HTTPAPI Ver 1.39 released 2018-03-09
NTLM Ver 1.4.0 released 2014-12-22
OS/400 Ver V7R3M0

2021-02-25-08.13.37.461000: http_persist_open(): entered
2021-02-25-08.13.37.500000: http_long_ParseURL(): entered
2021-02-25-08.13.37.500000: DNS resolver retrans: 2
2021-02-25-08.13.37.500000: DNS resolver retry : 2
2021-02-25-08.13.37.501000: DNS resolver options: x'00000136'
2021-02-25-08.13.37.501000: DNS default domain: MAPLESRUGS.COM
2021-02-25-08.13.37.501000: DNS server found: 172.16.1.42
2021-02-25-08.13.37.502000: DNS server found: 172.16.1.43
2021-02-25-08.13.37.502000: DNS server found: 172.16.1.42
2021-02-25-08.13.37.524000: Nagle's algorithm (TCP_NODELAY) disabled.

[Scott Klement]

The 28 byte fdset is to allow for 224 (8 bits x 28 bytes) descriptors to be tracked.

If you're running into errors with this, it most likely means that your job has too many descriptors open. Descriptors are used for IFS files, Sockets (network connections), and Pipes. Normally if there are more than 224 open in the same job at the same time, this indicates a problem that some program isn't closing them properly, and its eventually running out.

[Scott Klement]

I created a copy of HTTPAPI that doesn't use the 28-bite fd set concept. I don't have any easy way to test it, so I have no idea if it'll solve the problem you describe.

But, if you'd like to try it, you can download this experimental version of HTTPAPI here:

http://www.scottklement.com/httpapi/beta

The instructions for installing are the same as the normal version which can be found here:

http://www.scottklement.com/httpapi

If this works for you, please let me know. If feedback is good, I will make this the "new normal" for HTTPAPI.

[c0nfitty]

Thanks Scott, that makes a lot of sense.
This new process I'm working on contains a lot more web requests and IFS I/O's than everthing else running in production.

Http-api/1.43 worked great! I noticed that version 1.39 would give me the error mentioned above after around 10-15 requests. I just ran over 60 in a batch process with no errors. I'll post here again with any updates.