Jump to content
IGNORED

Working on advanced HTTP operations.


Recommended Posts

Currently working on advanced HTTP operations.

 

Currently, the AUX1 open values map as such:

 

4 = GET

8 = PUT

12 = GET with HEADERS (this is so you can write the headers requested)

13 = POST

 

Other verbs such as DELETE are handled through idempotent XIO commands.

 

You use the XIO 77 command to set a special channel mode for the HTTP adapter once opened:

 

XIO 77,#1,aux1,aux2,"N:"

 

aux1 should match the aux1 value used at open

 

aux2 is one of:

 

0 = DATA, reads/writes are HTTP BODY

1 = COLLECT_HEADERS, PRINTs of "HEADER" indicate the headers you wish to receive with GET_HEADERS

2 = GET_HEADERS, subsequent INPUTs will return header values in the order you COLLECTED them

3 = SET_HEADERS, subsequent PRINTs of "Header: Value" will put those headers into the request

4 = SEND_POST_DATA, subsequent PRINTs, add to the POST data sent.

 

With this, it should be possible to do every major type of web request needed.

 

The implementation of the HTTP adapter is currently here:

https://github.com/FujiNetWIFI/fujinet-platformio/blob/master/lib/network-protocol/HTTP.cpp

 

and the test harness I am currently using to test the GET and POST requests is here, it's implemented in Go:

https://gist.github.com/tschak909/5af74fec0e49c6071d924e7cb0b15680

 

and my last debug output after attempting to do a GET request with a query:

18:11:31.035 > sioNetwork::sio_process 0x4f 'O': 0x0c, 0x03
18:11:31.037 > sioNetwork::sio_open()
18:11:31.037 >
18:11:31.037 > ACK!
18:11:31.037 > <-SIO read 256 bytes
18:11:31.085 > ACK!
18:11:31.086 > sioNetwork::parseURL(N:HTTP://TMA-2:8000/?V=1&W=1)
18:11:31.086 > sioNetwork::parseURL transformed to (N:HTTP://TMA-2:8000/?V=1&W=1, HTTP://TMA-2:8000/?V=1&W=1)
18:11:31.088 > Parse and instantiate protocol: N:HTTP://TMA-2:8000/?V=1&W=1
18:11:31.088 > NetworkProtocol::ctor()
18:11:31.088 > sioNetwork::open_protocol() - Protocol HTTP opened.
18:11:31.088 > NetworkProtocolHTTP::mount(HTTP://TMA-2:8000/?V=1&W=1)
18:11:31.091 > fnHttpClient::begin "http://TMA-2:8000/?V=1&W=1"
18:11:31.091 > NetworkProtocolFS::resolve(/,/,)
18:11:31.091 > NetworkProtocolHTTP::stat(http://TMA-2:8000/?V=1&W=1)
18:11:31.091 > Resolved to http://TMA-2:8000/?V=1&W=1
18:11:31.093 > NetworkProtocolHTTP::open_file_handle()
18:11:31.093 > COMPLETE!
18:11:37.096 > 
18:11:37.097 > CF: 71 53 00 00 c4
18:11:37.097 > sioNetwork::sio_process 0x53 'S': 0x00, 0x00
18:11:37.097 > ACK!
18:11:37.097 > sioNetwork::sio_status_channel(0)
18:11:37.097 > PROTOCOL
18:11:37.097 > sio_status_channel() - BW: 65535 E: 1
18:11:37.099 > ->SIO write 4 bytes
18:11:37.099 > COMPLETE!
18:11:37.103 >
18:11:37.104 > CF: 71 52 7f 00 43
18:11:37.104 > sioNetwork::sio_process 0x52 'R': 0x7f, 0x00
18:11:37.104 > sioNetwork::sio_read( 127 bytes)
18:11:37.104 > ACK!
18:11:37.104 > NetworkProtocolFS::read_file(127)
18:11:37.105 > NetworkProtocolHTTP::read_file_handle(0x3fffadd0,127)
18:11:37.105 > fnHttpClient::GET
18:11:37.107 > 00012822 _perform
18:11:40.955 > 0001372b _perform status = 200, length = 23, chunked = 0
18:11:40.955 > C:\Users\thomc\.platformio\packages\framework-espidf\components\freertos\tasks.c:4811 (xTaskNotify)- assert failed!
18:11:40.956 >
18:11:40.956 > abort() was called at PC 0x40099cd4 on core 1
18:11:40.957 >
18:11:40.957 > Backtrace:0x40096f4e:0x3fff5bc0 0x40097635:0x3fff5be0 0x4009be35:0x3fff5c00 0x40099cd4:0x3fff5c80 0x40220e2e:0x3fff5ca0 0x40201b09:0x3fff5cc0 0x40201b51:0x3fff5ce0 0x4020843a:0x3fff5d00 0x40247061:0x3fff5d40 0x400dd3bf:0x3fff5d60 0x400dd434:0x3fff5d80 0x400dd832:0x3fff5da0 0x400e00f9:0x3fff5dc0 0x400e0202:0x3fff5e00 0x400d2787:0x3fff5e20 0x4009763d:0x3fff5e40
18:11:40.961 >
18:11:40.961 >
18:11:40.961 > ELF file SHA256: 8c1d89c82d4d2510
18:11:40.961 >
18:11:40.961 > Rebooting...

 

Fun times ahead, sigh.

-Thom

  • Like 1
Link to comment
Share on other sites

Small comment: I think there is a bug in the sioNetwork::~sioNetwork destructor.  It invokes clear() on all three buffers without checking first to see if they are nullptr.  Then, it checks for nullptr.  If any of the buffer pointers actually are null, the destructor will never get as far as the test/delete instructions.

 

Also, in C++ it is safe to delete a null pointer so the null pointer comparisons (as the code currently stands) are unnecessary.  I'd suggest setting the pointers to null after deleting them.

Edited by FifthPlayer
grammar
  • Like 2
Link to comment
Share on other sites

Regarding the line ending translation.  translate_receive_buffer in the case of CR/LF, will convert the CRs to ATASCII line endings then afterwards perform a blanket removal of LF characters in the buffer.  Is that a safe assumption to make?  If piece of text uses the LF character in a situation other than an actual line ending, the LF will be inadvertently removed.  I don't understand why the actions of both translate_receive_buffer and translate_transmit_buffer aren't symmetrical in this case.  (translate_transmit_buffer explicitly replaces ATASCII line endings with a CR+LF sequence, which seems both straightforward and legit).

 

Edited by FifthPlayer
  • Like 1
Link to comment
Share on other sites

I have just now gotten to the part where you talk about HTTP. 

 

Instead of using inheritance to solve the WebDAV problem, I suggest using delegation.  HTTP would inherit directly from NetworkProtocol rather than NetworkProtocolFS.  NetworkProtocolHTTP then acts as a wrapper for another instance of NetworkProtocol that is suited for the task it's being expected to perform at the time.  So if it needs to do WebDav, it instantiates a private instance of WebDavProtocol helper object (inheriting from NetworkProtocolFS) and routes through all outside calls to it.  If HTTP needs to do a simple data operation, it instantiates a data helper object (of the appropriate subclass of NetworkProtocolFS) and passes through to it.  The whole idea of the channelMode flag goes away, replaced by instances of classes that are suited to purpose.  Most importantly, it keeps the code for each use case separate and easier to get one's head around.

Edited by FifthPlayer
"delegation" is the term I was looking for
Link to comment
Share on other sites

5 minutes ago, FifthPlayer said:

I have just now gotten to the part where you talk about HTTP. 

 

Instead of using inheritance to solve the WebDAV problem, I suggest using delegation.  HTTP would inherit directly from NetworkProtocol rather than NetworkProtocolFS.  NetworkProtocolHTTP then acts as a wrapper for another instance of NetworkProtocol that is suited for the task it's being expected to perform at the time.  So if it needs to do WebDav, it instantiates a private instance of WebDavProtocol helper object (inheriting from NetworkProtocolFS) and routes through all outside calls to it.  If HTTP needs to do a simple data operation, it instantiates a data helper object (of the appropriate subclass of NetworkProtocolFS) and passes through to it.  The whole idea of the channelMode flag goes away, replaced by instances of classes that are suited to purpose.  Most importantly, it keeps the code for each use case separate and easier to get one's head around.

delegation might be the appropriate pattern indeed. I will chew on this.

-Thom.

 

Link to comment
Share on other sites

There is nothing magic about design patterns.  At some point people realized there were common ways of solving problems in code that they had found useful in multiple situations, and decided to give them names, so that people did not have to re-invent them and they could be used as shorthand between developers.

 

The decorator pattern wraps an object, and then customizes or adds behavior on top of the original object's functionality.  pseduo-code-ish, here's the idea:

 

class JSONHTTPConnection::HTTPConnection {

 

    HTTPConnection *wrapped_obj;

    public JSONHTTPConnection::JSONHTTPConnection(HTTPConnection *in) {

      // save a pointer to the wrapped obj

      wrapped_obj = in;

      // XXX other initialization

    }

 

    public int read(void* buf, int len) {

       success = wrapped_obj->read(buf, len);

       if (success) {

            //  XXX do JSON parsing here, build arrays with fields/values to pass back to the atari, etc

       }

       return success;

    }

}

 

In this manner the JSONHTTPConnection adds JSON parsing without cluttering up the main HTTP class.  The advantage over using inheritance is that decorators can be chained in a series at runtime.  The standard C++ input/output library works this way I think.

 

I don't know for sure if this will work or if it makes sense in this situation, it's a suggestion.  The more I think about it, inheritance alone is probably enough to solve the problem of layering JSON and XML on top of HTTP.

Edited by FifthPlayer
tweak the pseudocode a bit
Link to comment
Share on other sites

11 minutes ago, FifthPlayer said:

There is nothing magic about design patterns.  At some point people realized there were common ways of solving problems in code that they had found useful in multiple situations, and decided to give them names, so that they could be used as shorthand between developers.

 

The decorator pattern wraps an object, and then customizes or adds behavior on top of the original object's functionality.  pseduo-code-ish, here's the idea:

 

class JSONHTTPConnection::HTTPConnection {

 

    HTTPConnection *wrapped_obj;

    public JSONHTTPConnection::JSONHTTPConnection(HTTPConnection *in) {

      // save a pointer to the wrapped obj

      wrapped_obj = in;

      // XXX other initialization

    }

 

    public int read(void* buf, int len) {

       success = wrapped_obj->read(buf, len);

       if (success) {

            //  XXX do JSON parsing here, build arrays with fields/values to pass back to the atari, etc

       }

       return success;

    }

}

 

In this manner the JSONHTTPConnection adds JSON parsing without cluttering up the main HTTP class.  The advantage over using inheritance is that decorators can be chained in a series at runtime.  The standard C input/output library works this way I think.

 

I don't know for sure if this will work or if it makes sense in this situation, it's a suggestion.  Inheritance alone might be enough.

oh yeah, I do know. ;) I just don't keep all of that in my head. ;) (in fact, I actually keep very little in my head, I have worked in so many languages, that I have to have a screenful of code in what I'm working on, to snap my head into context)

 

(for those who don't know me, I've been a professional software developer now for 34 years, but rather unusual in that even though I've worked on some very insane things, I am entirely self taught)

 

-Thom

  • Like 2
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...