FastBasic Pre-processor (Windows)

[snicklin]

Hi,

 

This is the first released version of my FastBasic Pre-processor, v0.1.1.

 

This pre-processor should give you some more capabilities for developing with FastBasic on a PC. These capabilities include the ability to include other files in your main Basic file, text tag replacement and inserting line numbers into your output. The full capability list is in the included help file.

 

The idea is that you run this prior to running the FastBasic cross-compiler.

 

If you find any bugs, please let me know in your comments below, along with the text files that didn't work for you.

FBPP-0.1.1.zip

Edited December 3, 2021 by snicklin

[vitoco]

It's very interesting. Thanks for share it.

 

I have my own FastBasic preprocesor, which I developed to pack the source code into abbreviated form for my tenliners. It does not interact with the environment, but I use it to process binary data (chars in strings), manage constants, include external files, IF conditions useful to select palettes for NTSC and PAL versions or to activate debug code, remove comments and more.

[snicklin]

34 minutes ago, vitoco said:

It's very interesting. Thanks for share it.

 

I have my own FastBasic preprocesor, which I developed to pack the source code into abbreviated form for my tenliners. It does not interact with the environment, but I use it to process binary data (chars in strings), manage constants, include external files, IF conditions useful to select palettes for NTSC and PAL versions or to activate debug code, remove comments and more.

Thank you too for the kind comment!

 

Your preprocessor sounds very interesting. I really like your idea for including binary data. Is it something that you would also considering releasing onto here? I guess that it gives you a good advantage with the 10 liners

 

What language did you use to create your preprocessor? 

[vitoco]

My FastBasic preprocessor is a branch of the one I wrote during the development of my first tenliner games in TurboBasic XL about 6 years ago. These tools are perl scripts with lots of rules using regular expressions to convert reserved words into their abbreviations, replace escaped hexadecimal by their ATASCII char, remove unnecesary blank spaces and comments, join lines, and more, but they do not replace variable names with shorter ones like dmsc's TBXL Parser tool (which it is a real parser).

 

I won't release them, because they are not finished and stable products and I'm not proud of the code inside them. On every new game I have to hack the corresponding latest version to manage new situations (bugs), specially in the FastBasic version where I have to change the reserved words translation table because of new statements, functions or even changed abbreviations of the old ones. The ability to include external files was added this year during the development of The Children, because I used another perl tool to convert the puzzles from a text file into a compressed format in the form of ATASCII strings, so I could edit and test the puzzles without having to modify the source code each time (initially I did that: merge the files by hand using copy&paste in the text editor!!!).

 

You are right, there are some advantages on using these scripts:

• Simple editing of the source file using a free form text editor with syntax highlighting and auto indentation. No need to write and maintain obfuscated code by hand (like I initially did in my first two or three games).
• Conversion of the ASCII listing into ATASCII code ready to be loaded in the emulator using ENTER in TurboBasic XL and BASIC, or in FastBasic's IDE.
• Preview of the compressed listing to see how much free space is available for the next feature to be included in a game in development.

 

4 hours ago, snicklin said:

inserting line numbers into your output.

Which is the idea behind this? FB does not need line numbers.

 

[snicklin]

8 hours ago, vitoco said:

My FastBasic preprocessor is a branch of the one I wrote during the development of my first tenliner games in TurboBasic XL about 6 years ago. These tools are perl scripts with lots of rules using regular expressions to convert reserved words into their abbreviations, replace escaped hexadecimal by their ATASCII char, remove unnecesary blank spaces and comments, join lines, and more, but they do not replace variable names with shorter ones like dmsc's TBXL Parser tool (which it is a real parser).

 

I won't release them, because they are not finished and stable products and I'm not proud of the code inside them. On every new game I have to hack the corresponding latest version to manage new situations (bugs), specially in the FastBasic version where I have to change the reserved words translation table because of new statements, functions or even changed abbreviations of the old ones. The ability to include external files was added this year during the development of The Children, because I used another perl tool to convert the puzzles from a text file into a compressed format in the form of ATASCII strings, so I could edit and test the puzzles without having to modify the source code each time (initially I did that: merge the files by hand using copy&paste in the text editor!!!).

 

You are right, there are some advantages on using these scripts:

• Simple editing of the source file using a free form text editor with syntax highlighting and auto indentation. No need to write and maintain obfuscated code by hand (like I initially did in my first two or three games).
• Conversion of the ASCII listing into ATASCII code ready to be loaded in the emulator using ENTER in TurboBasic XL and BASIC, or in FastBasic's IDE.
• Preview of the compressed listing to see how much free space is available for the next feature to be included in a game in development.

 

Which is the idea behind this? FB does not need line numbers.

 

Perl gets criticised a lot, but it is the perfect language for this. I have done Perl development on and off for over 20 years.

 

The line numbers is so that you can add the line number from the source file, so that you can have a debugging routine.

 

Imagine

If a<6 then 'This should always happen

do something

else ' This should never happen

print "Error at line #LINE#"

endif

 

I understand your reasoning for not releasing your program, especially as it is in Perl, so there is no real way to completely hide the code. I won't release the code for FBPP as I am working on a proprietary Go library which it uses and I do not wish to release that part, as I may want to make it commercial later on. Edit: Note that this library isn't just for Atari stuff, but as a library for lots of general computing tasks, creating hashes, network comms, reading and writing Excel files, reading files etc etc.

 

I really like what you have been doing with your preprocessor. It is also interesting that we have both come up with preprocessors for different reasons. You for making your short games and me because I want to build tools to make it easier to build and debug more complex Basic programs.

 

 

 

 

Edited December 3, 2021 by snicklin

[vitoco]

39 minutes ago, snicklin said:

print "Error at line #LINE#"

I see. Good idea!

 

39 minutes ago, snicklin said:

I understand your reasoning for not releasing your program, especially as it is in Perl, so there is no real way to completely hide the code.

Being unable to hide the code is not the reason... it's because it will fail on many situations it cannot handle (because I haven't used such a feature in my games) and I'll receive many complains from potential users. For instance, TBXL version does not support every DATA data formats. As I said, I have to fix (add a feature) when I need it. I have a different version of one of the branches, stored in every tenliner development folder with specific fixes or tweaks. For the record: I just did a search in my tenliner projects folder and found 75 copies of the script (whichever branch), and 35 are unique!!! ? The oldest one is from february 2016, and the most recent version is from september 2021...

 

1 hour ago, snicklin said:

I won't release the code for FBPP as I am working on a proprietry Go library which it uses and I do not wish to release that part, as I may want to make it commercial later on.

That sounds great. I hope all goes well.

 

[snicklin]

6 hours ago, vitoco said:

For the record: I just did a search in my tenliner projects folder and found 75 copies of the script (whichever branch), and 35 are unique!!!

It sounds like you need one big super-script that does it all, by merging all of your work! I don't envy you doing that though!

 

I am looking for new ways that I can speed up development for FastBasic programmers. So if anyone wants to suggest a new feature, please do so, as long as it is realistic.

 

Although our preprocessors have different reasons for their creation, with different ambitions, I can see your idea of being able to embed binary data could be quite useful for the general FastBasic programmer. I might just steal this idea  I hope you're not offended!

 

[vitoco]

1 hour ago, snicklin said:

It sounds like you need one big super-script that does it all, by merging all of your work! I don't envy you doing that though!

The latest version of each branch includes all the previous version's features. Once I add a feature to one branch, I replicate it in the other when I have to use it. The exceptions are some mid versions where I had to keep a previous list of abbreviations for a previous version of FastBasic or when I used some special tweak for IF-THEN structure in BASIC or TBXL mixed with some other complex statement like DATA.

 

1 hour ago, snicklin said:

I can see your idea of being able to embed binary data could be quite useful for the general FastBasic programmer. I might just steal this idea  I hope you're not offended!

No problem... You can include any of my features in your tool. If you want, I can send you the latest version of the FB branch and an example to show what can be done.

 

[snicklin]

8 hours ago, vitoco said:

No problem... You can include any of my features in your tool. If you want, I can send you the latest version of the FB branch and an example to show what can be done.

Thank you! I didn't want to step on your toes here. I am working on adding a new INCBIN tag. It'll add everything into a big, long DATA statement. Luckily with the cross-compiled version, I don't have to worry about the length of the output line.

 

It would be interesting to see what you've been up to and to see some of the difficulties that you've had, and how you have overcome them.

[vitoco]

1 hour ago, snicklin said:

I am working on adding a new INCBIN tag. It'll add everything into a big, long DATA statement. Luckily with the cross-compiled version, I don't have to worry about the length of the output line.

I'm not completely sure about your INCBIN tag if you will be feeding the cross-compiler. FastBasic already have a special syntax for the cross-compiler to do exactly that, for both byte and word arrays. From DATA definition in the doc:

 

Quote

 

Loading data from a file

 

The cross-compiler also supports loading data from a file directly into the program, using the BYTEFILE (abbreviated BYTEF.) and WORDFILE (abbreviated WORDF. or simply F.) types and a file name enclosed in double quotes.

Example:

  DATA img() bytefile "img.raw"
  DATA pos() wordfile "pos.bin"

The compiler will search the file in the same folder than the current basic source.

 

 

As the target of the output file of my preprocessor is the FastBasic IDE in the Atari, what I've added to it is the ability to include more source lines from an external file, which it could happen to include binary data in the form of strings with escaped code (\HH hex literals), and then they are translated into ATASCII as regular lines with strings. There is no direct conversion of a binary file into a DATA statement (but I like the idea).

 

Another idea I have had many times is to include a filter to map text strings into their corresponding internal code and/or shift them to another playfield color (for graphics modes 1 and 2 lines, or inverse video), but I think I never implemented it directly inside the preprocessor. It could be a nice feature to yours.

 

[snicklin]

10 hours ago, vitoco said:

I'm not completely sure about your INCBIN tag if you will be feeding the cross-compiler.

This is what happens when I haven't actually programmed in FastBasic for many months, I forgot some of its functionality!

 

10 hours ago, vitoco said:

Another idea I have had many times is to include a filter to map text strings into their corresponding internal code and/or shift them to another playfield color (for graphics modes 1 and 2 lines, or inverse video), but I think I never implemented it directly inside the preprocessor. It could be a nice feature to yours.

That is a good idea. It's not something that I use myself (Graphics mode 1 and 2, I despise those graphics modes!), but I can see this feature being useful for another user. Thanks! This reminds of keyboard scan codes also.

[telengard]

On 12/3/2021 at 9:33 PM, snicklin said:

It sounds like you need one big super-script that does it all, by merging all of your work! I don't envy you doing that though!

 

I am looking for new ways that I can speed up development for FastBasic programmers. So if anyone wants to suggest a new feature, please do so, as long as it is realistic.

 

Although our preprocessors have different reasons for their creation, with different ambitions, I can see your idea of being able to embed binary data could be quite useful for the general FastBasic programmer. I might just steal this idea  I hope you're not offended!

 

This is cool!  I stumbled onto this post looking for a means to edit FB on my Linux box for cross compiling.  Is the code "portable" such that it could be built for Linux?  Also, one of the things I really would like to find is a tool that can convert normal text ATASCII inline, I'm specifically thinking about inverse etc and converting the unicode to viewable text.  You mentioned feature requests, is this something you could add?

something like <i>text</i> for inverted text might be a good solution, and the pre-processor could convert to the appropriate ATASCII.  There would need to be one for normal text as well.

This is what I see now in my editor (vi)
 

I'm new to this so it's possible I'm not explaining things just right, but the ability to see and type regular ASCII chararacters in a PC editor and have them work in BASIC print statements after some text post processing would be awesome!

[snicklin]

On 1/23/2023 at 3:07 AM, telengard said:

This is cool!  I stumbled onto this post looking for a means to edit FB on my Linux box for cross compiling.  

Hi Telengard! Thanks for commenting, I got the impression that nobody was using this!  As such I haven't looked at this code for 18 months and have forgotten a lot of it!

On 1/23/2023 at 3:07 AM, telengard said:

Is the code "portable" such that it could be built for Linux?  

Yes and no. So this program in written in Go (a.k.a. Golang). I am not sure how much you know about the language, but it's a cool new-ish language from Google which is really good for command line utilities, both across Windows and Linux (and some other OS's I believe).

 

As I am a Windows user, so far, all my libraries that I have been writing have been written for Windows. This is mainly concerning getting folder/directory separators, i.e. the forwards slash or using a backslash. It shouldn't be too difficult to convert to Linux, however I haven't got a running Linux system (I could dig one out, but it'll be hassle) and I don't have a VM set up yet. I do have the aim to make my code portable across the OS's, but that's not the only complication.

 

The main complication here is that I am writing these tools as exercises for myself to build up a great library of Go libraries. I want to use what I am developing to potentially make commercial at some point. For that reason, I will not be releasing any source code as that would compromise me later down the line. Of course, I could do Linux builds myself, but that would require me to set things up. So at the moment, I will stick to Windows builds. You should however be able to use this Windows command line tool together with WINE. I've not tested it with WINE, but in theory it should work.

 

I also need to port the program over to use the new versions of my libraries. I've got some time off at the moment with it being Australia Day, so I have commenced looking into this.

 

On 1/23/2023 at 3:07 AM, telengard said:

something like <i>text</i> for inverted text might be a good solution, and the pre-processor could convert to the appropriate ATASCII.  There would need to be one for normal text as well.

This is what I see now in my editor (vi)
 

I'm new to this so it's possible I'm not explaining things just right, but the ability to see and type regular ASCII chararacters in a PC editor and have them work in BASIC print statements after some text post processing would be awesome!

I believe I understand what you're asking for here. You want to be able to get some of the international characters (which you can use on a Linux or Windows PC) to be usable in a FastBasic program? I think that this could be implemented with a tag something like this (not implemented yet):

 

10 PRINT "#ASCII#José Mourinho# is a terrible football manager"

20 GOTO 10

 

Then it might need to spit out something like:

10 PRINT "Jos",CHAR(20)," Mourinho is a terrible football manager"

20 GOTO 10

 

I could see there being complications with breaking line length limits, but that might just need to be something that has to be accepted if someone writes too much output.

 

I will look into all of this over the next few days. No promises though as I need to relearn my own code...!

 

Thanks for the suggestion.

[dmsc]

Hi!

On 1/26/2023 at 12:50 AM, snicklin said:

I believe I understand what you're asking for here. You want to be able to get some of the international characters (which you can use on a Linux or Windows PC) to be usable in a FastBasic program? I think that this could be implemented with a tag something like this (not implemented yet):

 

10 PRINT "#ASCII#José Mourinho# is a terrible football manager"

20 GOTO 10

 

Then it might need to spit out something like:

10 PRINT "Jos",CHAR(20)," Mourinho is a terrible football manager"

20 GOTO 10

In this case, it is better to transform the string to this:

DO
 PRINT "Jos"$14" Mourinho is a terrible football manager"
LOOP

 

The "$" syntax inside strings let you embed any character in a string without runtime overheads, the compiler just builds the string including the character (in hex).

 

I think that the OP wants to embed inverse-video characters, this is currently awkward as it looks like:

? " "$C8$C5$Cc$Cc$Cf" "

 The above will print "HELLO" to the screen.

 

Have Fun!

[snicklin]

14 hours ago, dmsc said:

Hi!

In this case, it is better to transform the string to this:

Hi again Daniel. Many thanks, I had forgotten the exact syntax.

 

14 hours ago, dmsc said:

I think that the OP wants to embed inverse-video characters, this is currently awkward as it looks like:

? " "$C8$C5$Cc$Cc$Cf" "

 

 

Ah yes, true. I'll think something up.

 

So I managed to port it over to using my newer libraries. It compiles. But I haven't tested the output executable yet. This I plan to do next.

 

Once that is up and running, I'll look into adding new features.

[snicklin]

On 1/23/2023 at 3:07 AM, telengard said:

 

This is cool!  I stumbled onto this post looking for a means to edit FB on my Linux box for cross compiling.  Is the code "portable" such that it could be built for Linux?  Also, one of the things I really would like to find is a tool that can convert normal text ATASCII inline, I'm specifically thinking about inverse etc and converting the unicode to viewable text.  You mentioned feature requests, is this something you could add?

something like <i>text</i> for inverted text might be a good solution, and the pre-processor could convert to the appropriate ATASCII.  There would need to be one for normal text as well.

This is what I see now in my editor (vi)
 

I'm new to this so it's possible I'm not explaining things just right, but the ability to see and type regular ASCII chararacters in a PC editor and have them work in BASIC print statements after some text post processing would be awesome!

 

I haven't managed yet to do anything with international characters, however I have now added the ability to use inverse video text.

 

You should use the #INVERSE# tag as outlined in the help file included.

 

I've not tested it from a FB point of view, only that it is producing output text in the format that @dmsc has specified above.

 

It's in BETA status as I haven't extensively tested it as of yet. Could you take a look and see if it does what you want?

 

There will be a limitation that it will only handle those characters that map to the same values in ascii as in atascii (i.e. 0-9, a-z and A-Z should be fine, but other characters may play up).

 

fbpp_0.3.0.BETA.zip

[telengard]

On 2/27/2023 at 7:57 AM, snicklin said:

 

I haven't managed yet to do anything with international characters, however I have now added the ability to use inverse video text.

 

You should use the #INVERSE# tag as outlined in the help file included.

 

I've not tested it from a FB point of view, only that it is producing output text in the format that @dmsc has specified above.

 

It's in BETA status as I haven't extensively tested it as of yet. Could you take a look and see if it does what you want?

 

There will be a limitation that it will only handle those characters that map to the same values in ascii as in atascii (i.e. 0-9, a-z and A-Z should be fine, but other characters may play up).

 

fbpp_0.3.0.BETA.zip 2.25 MB · 1 download

Hi, thank you for implementing that!

 

I tested it out, and unless I'm doing something wrong (quite possible), it's not doing the inversing of the text.

 

PRINT #INVERSE#This is a Test#

COUNT = 0

DO
	COUNT = COUNT + 1
LOOP

 

After pre-processing

 

PRINT " "$54$68$69$73$20$69$73$20$61$20$54$65$73$74" "

COUNT = 0

DO
	COUNT = COUNT + 1
LOOP

 

Attached is a screenshot of it running.  I'm cross compiling on Linux and running the pre-processor on Windows.

 

 

[snicklin]

17 hours ago, telengard said:

Hi, thank you for implementing that!

 

I tested it out, and unless I'm doing something wrong (quite possible), it's not doing the inversing of the text.

 

PRINT #INVERSE#This is a Test#

COUNT = 0

DO
	COUNT = COUNT + 1
LOOP

 

After pre-processing

 

PRINT " "$54$68$69$73$20$69$73$20$61$20$54$65$73$74" "

COUNT = 0

DO
	COUNT = COUNT + 1
LOOP

 

Attached is a screenshot of it running.  I'm cross compiling on Linux and running the pre-processor on Windows.

 

 

Ahh, this is me misunderstanding the required FastBasic above. I will take a look at this in the next few days. Silly me. 

 

That's the problem with programming all day long and then picking up programming as a night time hobby also. I am going from a long day of 4D, PHP and Javascript to a night of Go and FastBasic tired....

[+Stephen]

2 hours ago, snicklin said:

That's the problem with programming all day long and then picking up programming as a night time hobby also. I am going from a long day of 4D, PHP and Javascript to a night of Go and FastBasic tired....

So it's not just me then - I've been going from C# and a grueling T-SQL to MySQL conversion project in the day, to 6502 and low-level VBXE coding in evenings.  Then I wonder why I am so tired all the time.

[ClausB]

But the evening work is more satisfying, no?

[+Stephen]

1 hour ago, ClausB said:

But the evening work is more satisfying, no?

Very much so.

[telengard]

4 hours ago, snicklin said:

Ahh, this is me misunderstanding the required FastBasic above. I will take a look at this in the next few days. Silly me. 

 

That's the problem with programming all day long and then picking up programming as a night time hobby also. I am going from a long day of 4D, PHP and Javascript to a night of Go and FastBasic tired....

Please, don't feel you need to do this at all or in any sorta time frame.  I'm also a day/night dev and can relate, finding the energy at night to do the fun stuff with all the other things going on can be a real challenge.

[snicklin]

On 3/2/2023 at 12:51 AM, Stephen said:

So it's not just me then - I've been going from C# and a grueling T-SQL to MySQL conversion project in the day, to 6502 and low-level VBXE coding in evenings.  Then I wonder why I am so tired all the time.

I do a little C# in my job, just to maintain an existing codebase, and it is horrendous. So hats off to you for doing that. Plus all the other stuff in your day job. Oh, and the low level VBXE work too!

 

On 3/2/2023 at 1:57 AM, ClausB said:

But the evening work is more satisfying, no?

Very much so, ditto. When it is your choice of programming, it's always better!

 

On 3/2/2023 at 3:37 AM, telengard said:

Please, don't feel you need to do this at all or in any sorta time frame.  I'm also a day/night dev and can relate, finding the energy at night to do the fun stuff with all the other things going on can be a real challenge.

Ah that's alright, I'll sort it out as and when I feel like it. I am looking at it at the moment, but it may be done tonight, might be next week, maybe later, but thanks for the kind words also.

 

 

 

Edited March 3, 2023 by snicklin

[snicklin]

OK, here is the latest version. Inverse should work in the same way mentioned before, except that it doesn't print out extra spaces and should show inverse text. Please note that it won't work for any characters that don't map from ascii->atascii directly. It should be good enough for general strings though.

 

 

 

 

fbpp.zip

[telengard]

It works, thank you so much!  This will be very handy.

 

Can confirm it also does work well in WSL.

Edited March 7, 2023 by telengard