Camel99 Forth Information goes here

[+TheBF]

I published a video to demonstrate what it's like to use CAMEL99 Forth on Classic99.  I am pretty happy with the usability of the system now. Compile times are pretty fast with the changes I made recently. 

Just need to decide on the best way to make a file editor.

 

https://youtu.be/__PtCJ473Zk

 

[+TheBF]

A good use for Low RAM

 

The way Camel99 Forth is structured the 8K RAM at >2000 is treated as a HEAP. This is just a fancy name for memory that has no specific purpose but can be used by your program as needed.  For example when Camel99 Forth kernel reads a file of source code it grabs an 82 byte buffer from the HEAP to read each line of the file into memory.  This is done simply by adding 82 to the heap pointer variable "H".  When it's finished it restores the heap pointer by subtracting 82 from H. Simple.

 

In this example we use the MALLOC library which makes this feature a little easier to use and it only takes a couple of lines of code.

: MALLOC ( n -- addr ) H @  SWAP H +! ;
: MFREE  ( n -- ) NEGATE H +! ;

 

So the "good use" is to use the HEAP memory space to compile the Forth Assembler and Tools.

I made COMPILE-LOW and COMPILE-HI and it seems to work nicely.

 

The method is:

1. Save the current dictionary pointer DP in a temp variable
2. MALLOC a big buffer in the HEAP, move Forth's dictionary pointer to the buffer
3. Compile the tools that you want to use
4. The new position of the dictionary pointer is saved as the HEAP pointer
   (This marks that memory as not available, but makes the remainder available)
5. Put the Forth dictionary pointer back where it used to be in High RAM.

 

After that we have 15K of program space for our program and about 1/2 the HEAP is used up.

\ LOWTOOLS  puts ASSEMBLER and tools into LOW RAM  Feb 27 2020  BFox
NEEDS MALLOC FROM DSK1.MALLOC
HEX
VARIABLE SAVEDP
: COMPILE-LOW ( -- )
    HERE  SAVEDP !    \ save the dictionary pointer
    1E00 MALLOC DP !  \ get a big buffer, DP(dictionary) points to the buffer
;

: COMPILE-HI  ( -- )
    HERE H !          \ give back what we didn't use to the HEAP
    SAVEDP @ DP !     \ restore DP back to original address
;
COMPILE-LOW
  INCLUDE DSK1.ASM9900
  INCLUDE DSK1.TOOLS
COMPILE-HI
.FREE
HEX

The thing I need to figure out now is how to re-link the dictionary to remove the tools from the dictionary while still keeping my program.

With that feature the assembler could be loaded as needed for compiling a project, but removed from the final program.

More thinking required...

 

Edited February 28, 2020 by TheBF
typo

[+Lee Stewart]

21 minutes ago, TheBF said:

The thing I need to figure out now is how to re-link the dictionary to remove the tools from the dictionary while still keeping my program.

With that feature the assembler could be loaded as needed for compiling a project, but removed from the final program.

More thinking required...

 

Would you not copy the link in the first word of the temporary block of words (assembler and/or other tools) to the link in the first program word? This should link the first program word to the last word in the dictionary before the temporary block was compiled.

 

...lee

[+TheBF]

25 minutes ago, Lee Stewart said:

 

Would you not copy the link in the first word of the temporary block of words (assembler and/or other tools) to the link in the first program word? This should link the first program word to the last word in the dictionary before the temporary block was compiled.

 

...lee

That sounds correct.  I need to make a picture to see it clearly in my head.   I think I can use LATEST to get the links before I start compiling the temporary code and then again when I return to high memory.  I will have to create some more temp variables to remember those things I guess.

My old HsForth system has the words PERMANENT and TRANSIENT which are nice sounding. 

Any thoughts on naming?

 

[+Lee Stewart]

You are not using vocabularies, are you?

 

...lee

[+TheBF]

33 minutes ago, Lee Stewart said:

You are not using vocabularies, are you?

 

...lee

No. So it's a little less complicated in that way.

[+TheBF]

So this works ... mostly

\ RELINK

VARIABLE TRANSLINK
: TRANSIENT  ( -- ) LATEST @ TRANSLINK ! ;
: RELINK     ( -- ) TRANSLINK @  LATEST @ NFA>LFA ! ;

But if I say:

: WORD1 ;

TRANSIENT

: WORD2 ;
: WORD3 ;
: WORD4 ;

RELINK

I still see WORD4 in the dictionary.

 

[+Lee Stewart]

I know in Camel99 Forth that you tick a word to get its execution token (xt). Can you then get the label field address (lfa) from the xt as can be done in fbForth through the parameter field address (pfa)? If so, the word that begins the permanent block (PROGRAM, say) could be ticked by a redefined RELINK to place TRANSLINK’s contents into PROGRAM’s lfa:

RELINK PROGRAM

 

...lee

[+TheBF]

I just sat back at the machine 5 minutes ago.  I think I understand what you are getting at and it can be done.

 

However for conversation, this is one of the weaknesses of the Camel Forth Header.

*translation layer: Fig LFA = Camel Forth NFA   

 

Here is structure.

\     D7           D0
\     +---------------+
\     |               |   byte 1      \ LINK FLD contains NFA of previous word
\     |-    link     -|
\     |               |   byte 2
\     +-------------+-+
\     |      0      |P|   byte 3     \ P - Precedence bit, equals 1 for an IMMEDIATE word
\     +-+-----------+-+
\     |S|   length    |   byte 4     \ S - Smudge bit, used to prevent FIND from finding this word.
\     +-+-------------+
\     |               |   byte 5
\     |-    name     -|
\     |               |   ... to byte N
\     ~~~~~~~~~~~~~~~~~
\     |               |   byte n+1
\     |-  code field -|
\     |               |
\     +---------------+
\

You can get from the name field to the link easily.

You can get from the link field to the name easily.

You can get from the name field to the code field easily.

 

Getting from the code field back to the name field is awkward because it is a variable length.

I made a way to do it but it is a loop that reads backwards looking for the precedence field. I mask off bit 1 and look for the zero.

It works but its ugly. (to me)

 

The real solution is something they have discussed in comp.lang.forth called FIND-NAME. I wrote one today in the hopes of adding FORGET.

30 MALLOC CONSTANT TEMP$

: FIND-NAME  ( addr len -- nfa )  \ 4.4x SLOWER than ASM (FIND)
             TEMP$ PLACE
             LATEST @
             BEGIN
                DUP 1+  TEMP$ COUNT S=
                0= IF  EXIT  THEN
                NFA>LFA @ DUP
             WHILE REPEAT ;

I It's not pretty with the TEMP$ but it works.  I need to re-work it to use R stack but I need a 2R@ to do it nicely.

 

(BTW I noticed that FORGET has made it back into the new "kind of a standard" Forth 2012. I always liked that word. Much cooler than MARKER._

 

I fixed the problem in my first RELINK test just now. I forgot to re-adjust LATEST to point to the new beginning of the linked list.

: RELINK     ( -- ) 
             TRANSLINK @ DUP  LATEST @ NFA>LFA !  
             LATEST !  ;

So I have successfully RELINKed that simple example. It's a beginning.

Thanks for the discussion. I really appreciate it.

 

 

 

[+TheBF]

I fixed FIND-NAME so it doesn't need a buffer.  It scans through 378 words in 0.26 seconds.

S= ( addr addr len --  -1|0|1 )  is a string comparison code word in Camel Forth.

 

Combined with the learning in RELINK, I should be able to use this to make FORGET.

\ : 2OVER  3 PICK  3 PICK ;  ( faster inline)

: FIND-NAME ( addr len -- nfa )
             LATEST @
             BEGIN
                DUP 1+ 3 PICK 3 PICK S=
                0= IF  NIP NIP EXIT THEN 
                NFA>LFA @ 
                DUP
             0= UNTIL
             NIP NIP ;

 

[+TheBF]

It was easier than I thought to make FORGET.

I factored out some things differently for clarity if I ever come back to this and the new words are useful in their own right.

\ FORGET    erase dictionary up a given word
NEEDS S=     FROM DSK1.COMPARE
: 2OVER    ( a b c d -- a b c d a b) 3 PICK  3 PICK ;
: 2NIP     ( a b c -- c)  NIP NIP ;

: FIND-NAME ( addr len -- nfa ) \ nfa is "name field address"
           LATEST @
           BEGIN
              DUP 1+ 2OVER S=
              0= IF  2NIP  EXIT THEN
              NFA>LFA @   ( next name in dictionary)
              DUP
           0= UNTIL
           2NIP  ;

: FORGET ( <text> )
           BL WORD COUNT  FIND-NAME
           DUP 0= ?ERR
           DUP NFA>LFA DUP  @ LATEST !  \ relink the words
           ( lfa )  DP ! ;              \ recover the memory

 

EDIT:  Added  line in FORGET to recover dictionary space. Removed NEXTNAME as a word. It was not useful enough. 

That's enough for tonight.

Edited February 29, 2020 by TheBF

[+Lee Stewart]

Why do you need to define FIND-NAME ? Do you not have ' and ['] to find names starting with LATEST ? Once you get the execution token, then traverse the name backwards to the precedence bit and correct to the nfa. H-m-m-m... Maybe, instead, you should redefine ' and ['] in terms of FIND-NAME . But, I digress...

 

...lee

[+TheBF]

This is so weird. Your message came in 5 minutes after I sat down here.  Are you watching me? ?

 

You are correct. I made a word called CFA>NFA the scans backwards in the header to get to the precedence bit and then adds 1. 

I may give that a go as well. 

 

BTW in your wisdom you have succinctly summarized endless discussions in various fora on FIND and how it needs be replaced with FIND-NAME because FIND-NAME is a better factor with which to build FIND. And FIND-NAME does not have to copy the string to HERE before it's interpreted so it is more efficient ... yada yada yada.

 

I just made re-linking work manually and it is pretty simple as we thought.  I just need to give things names and I will put it up shortly for your perusal.

 

( I wanted an excuse to make FIND-NAME and then code in assembler as a factor for FIND.

  But your suggestion of simply using tick and counting backwards is probably just a good.

  Luckily I don't get paid for doing this. )

 

 

 

 

[+TheBF]

FORGET is definitely smaller this way.

\ FORGET    erase dictionary up a given word

: CFA>NFA ( cfa -- nfa | 0 ) 2- BEGIN   1- DUP C@ 0FE AND  0= UNTIL 1+  ;

: FORGET   '   CFA>NFA  NFA>LFA DUP @ ( nfa) LATEST ! ( lfa ) DP ! ;

 

Edited March 3, 2020 by TheBF
Removed extra DUP in FORGET

[+Lee Stewart]

I do not know whether you really need it, but figForth (I think it harks back that far) used the user variable FENCE to hold the address of the last word that cannot be forgotten, i.e., it would only allow words between the address in FENCE and HERE to be forgotten by FORGET . The default for FENCE is the last word in the kernel. TI Forth and fbForth will not allow any part of the kernel to be forgotten regardless of the value in FENCE . 

 

...lee

[+TheBF]

 

3 minutes ago, Lee Stewart said:

I do not know whether you really need it, but figForth (I think it harks back that far) used the user variable FENCE to hold the address of the last word that cannot be forgotten, i.e., it would only allow words between the address in FENCE and HERE to be forgotten by FORGET . The default for FENCE is the last word in the kernel. TI Forth and fbForth will not allow any part of the kernel to be forgotten regardless of the value in FENCE . 

 

...lee

I remember that and I will have been cogitating on it as well. 

Working code coming shortly for the re-linking.

 

[+TheBF]

So this works but I had to change the kernel slightly.  Like FbForth and PAD I have been using this HEAP idea alot just to kick the tires.

I put the scroll buffer in a "heap-pad" ie: un-allocated memory in low RAM.  But... I also was allocating and de-allocating a buffer for INCLUDED in the kernel that loads source code.  The fix for this low ram compiling was to lock down the line buffer for INCLUDED so my heap now starts at >2050.

Therefore this code will not work in v2.5 only this new one I have not released yet.

 

But it was as simple as you said once I understood what was where.

\ LOWTOOLS  puts ASSEMBLER and tools into LOW RAM  Feb 27 2020  BFox
NEEDS MALLOC FROM DSK1.MALLOC

HEX
VARIABLE SAVEDP

 LATEST @ CONSTANT KEEP  \ remember latest name field address

\ set up low ram compiling .............
  HERE SAVEDP !          \ save the dictionary pointer.
  1E00 MALLOC DP !       \ get a big buffer, dictionary points to the buffer

\ =======================================================================
\ *INSIGHT*
\ SAVEDP holds the LINK field of the 1st new word we will create in HI RAM
\ =======================================================================
  INCLUDE DSK1.TOOLS
  INCLUDE DSK1.ASM9900

: REMOVE-TOOLS ( -- )
         KEEP SAVEDP @ !  \ relink the dictionary
         2050 H ! ;       \ init-the heap. (80 byte buffer is at >2000)

\ setup hi ram compiling ..............
 HERE H !          \ give back what we didn't use to the HEAP
 SAVEDP @ DP !     \ restore DP back to original address

\ test words will remain in dictionary after REMOVE-TOOLS
: WORD1 ."  word1" ;
: WORD2 ."  word2" ;
: WORD3 ."  word3" ;
: WORD4 ."  word4" ;

.FREE
HEX

 

Edited March 1, 2020 by TheBF
typos

[+Lee Stewart]

That is very cool—except for that one time you decide to compile something other than a word definition as the first item in high memory. That is why I suggested ticking the first word in which to patch the lfa. Of course, you could install a no-op marker word as the first item of business for the new program.

 

...lee

[+Lee Stewart]

5 minutes ago, Lee Stewart said:

That is why I suggested ticking the first word in which to patch the lfa.

 

Actually, I should not take credit for that much insight.  I did not think of that reason until after I figured out what you were doing in the “relink the dictionary” line of REMOVE-TOOLS .

 

...lee

[+TheBF]

Ya I was considering how I could make a word that compiles a word when executed for that purpose.  I think I can do it, but I came out of the basement to spend some time with my wife.   

 

Higher callings.

 

Thanks again for the support.

 

 

[+TheBF]

Ok so at the risk of sounding stupider than I already do...

 

I can't think of what I could compile after I do a REMOVE-TOOLS that would make a problem except maybe trying to compile raw numbers with no CREATE <word> first, which I would never do, like this:

DEAD , BEEF , 

 

Every other thing is built on the word HEADER like this:

\ ========================================================================
\ D I C T I O N A R Y   C R E A T I O N

: HEADER, ( addr len --)
            ALIGN
            LATEST @ ,
            0 C,
            HERE LATEST !
            S, ;

: HEADER   BL PARSE-WORD HEADER, ;

\ ========================================================================
\ T A R G E T   S Y S T E M   D E F I N I N G   W O R D S

\                    text    runtime-action   parameter
\ -------------------------  --------------- -----------
: CONSTANT  ( n --)  HEADER  COMPILE DOCON     COMPILE, ;
: USER      ( n --)  HEADER  COMPILE DOUSER    COMPILE, ;
: CREATE    ( -- )   HEADER  COMPILE DOVAR              ;
: VARIABLE  ( -- )   CREATE                  0 COMPILE, ;

 

And even ':' is defined with HEADER

 X: :         !CSP  HEADER (:NONAME)  ;X

So in every case the new link field will be at the high RAM address and will be stuffed with the contents of LATEST to hook the new word into the linked list.

What am I missing?

 

I am reminded of Desi Arnez in I Love Lucy.  "You got some 'splainin' to do"  

 

 

[+Lee Stewart]

3 hours ago, TheBF said:

Ok so at the risk of sounding stupider than I already do...

 

Ridiculous! 

 

3 hours ago, TheBF said:

So in every case the new link field will be at the high RAM address and will be stuffed with the contents of LATEST to hook the new word into the linked list.

What am I missing?

 

I am reminded of Desi Arnez in I Love Lucy.  "You got some 'splainin' to do"  

 

Exactly what you would never do I have done quite often in fbForth with DATA[ ... ]DATA , a construct you probably do not have in Camel99 Forth. It uses up dictionary space and leaves its address and cell count on the stack. If I were to adapt REMOVE-TOOLS to fbForth, I would likely need to install that initial dummy word.

 

...lee

[+TheBF]

51 minutes ago, Lee Stewart said:

 

Ridiculous! 

 

 

Exactly what you would never do I have done quite often in fbForth with DATA[ ... ]DATA , a construct you probably do not have in Camel99 Forth. It uses up dictionary space and leaves its address and cell count on the stack. If I were to adapt REMOVE-TOOLS to fbForth, I would likely need to install that initial dummy word.

 

...lee

Ah yes. That makes sense now. 

I have been noodling on perhaps using the HEADER,  word to make a dummy word as part of REMOVE tools.  It would need to have a way to EXIT right back to Forth. 

But just adding a dummy word manually in the file  would be just as easy since I plan to INCLUDE DSK1.LOWTOOLS  as a "package".

 

 

 

[+Lee Stewart]

4 hours ago, TheBF said:

Ah yes. That makes sense now. 

I have been noodling on perhaps using the HEADER,  word to make a dummy word as part of REMOVE tools.  It would need to have a way to EXIT right back to Forth. 

But just adding a dummy word manually in the file  would be just as easy since I plan to INCLUDE DSK1.LOWTOOLS  as a "package".

 

Yeah—perhaps a one-character word using an unused character like ‘~’ or ‘`’ would take up the least space:

: `  ;     \ dummy starting word

 

...lee

[+TheBF]

All this work has also given me the tools to make a word from HsForth called BEHEAD.  

It allows you to compile a bunch of code and then remove the names of some of the words so that they can't be seen.

Not something I urgently need but I know how to do it now. 

 

The other feature I have used in MaxForth was the word names  could be compiled in a separate part of memory so you could strip them all away when the project was finished. This made the EPROM image as much as 30% smaller. 

 

That would be a cool one if I also develop a save to EA/5 image to go with it. You could build some very tight programs in Forth that way. Probably as small as ALC.

 

Some much code so little time...