What happened to the Emscripten branch?

Emscripten, WASM, and asm.js related questions
Post Reply
angros47
Posts: 2321
Joined: Jun 21, 2005 19:04

Re: What happened to the Emscripten branch?

Post by angros47 »

Perhaps I figured it out. Looks like the assembly intermediate file created by clang has something wrong in it. Usually, with emcc, clang outputs directly an object file, so there is no need for the assembly file (likely in the past, when V1ctor made the port, it used llvm assembly, but nowaday it doesn't). Unfortunately, FreeBasic uses a two step compiling when using the C emitter: from basic to C, from C to assembly, and only then from assembly to .o.
I modified the compiler to skip the intermediate passage, and compile directly from C to object code, on the emscripten target (other targets should be unaffected). The issues reported by VANYA seem to be gone, in that way. It is not possible to get the .s file, now, only the .c file, but since the .s file didn't work well (and since I doubt anyone would need an intermediate assembly file for something that will be compiled to Javascript) I guess it should work.

Well, this is another patch:

Code: Select all

--- fbc.bas	2020-08-31 20:48:38.000000000 +0200
+++ src/compiler/fbc.bas	2020-11-11 21:49:25.602909281 +0100
@@ -133,7 +133,7 @@
 static shared as zstring * 16 toolnames(0 to FBCTOOL__COUNT-1) = _
 { _
 	"as", "ar", "ld", "gcc", "llc", "dlltool", "GoRC", "windres", "cxbe", "dxe3gen", _
-	"llvm-as", _
+	"emcc", _
 	"emar", _
 	"emcc", _
 	"emcc"  _
@@ -2634,7 +2634,7 @@
 	if( fbGetOption( FB_COMPOPT_TARGET ) <> FB_COMPTARGET_JS ) then
 		ext = @".asm"
 	else
-		ext = @".llvm"
+		ext = @".o"
 	end if
 
 	if( stage = 1 ) then
@@ -2975,12 +2975,15 @@
 			ln += "-fPIC "
 		end if
 
-		ln += "-S -nostdlib -nostdinc -Wall -Wno-unused-label " + _
-		      "-Wno-unused-function -Wno-unused-variable " 
 			  
 		if( fbGetOption( FB_COMPOPT_TARGET ) <> FB_COMPTARGET_JS ) then
+			ln += "-S -nostdlib -nostdinc -Wall -Wno-unused-label " + _
+			      "-Wno-unused-function -Wno-unused-variable " 
 			ln += "-Wno-unused-but-set-variable "
 		else
+			'if Emscripten is used, we will skip the assembly generation and compile directly to object code
+			ln += "-c -nostdlib -nostdinc -Wall -Wno-unused-label " + _
+			      "-Wno-unused-function -Wno-unused-variable " 
 			ln += "-Wno-warn-absolute-paths -s ASYNCIFY=1 -s RETAIN_COMPILER_SETTINGS=1 "
 		end if
 
@@ -3152,6 +3155,13 @@
 			end if
 		endif
 	end if
+
+	if( fbGetOption( FB_COMPOPT_TARGET ) = FB_COMPTARGET_JS ) then
+		'We will skip assemble stage, since it is already performed by Emscripten
+		function = TRUE
+		exit function
+	end if
+
 	ln += """" + hGetAsmName( module, 2 ) + """ "
 	ln += "-o """ + *module->objfile + """"
 	ln += fbc.extopt.gas
For VANYA, this is the file to replace: https://www.mediafire.com/file/uxzyq6p1 ... c.bas/file
VANYA
Posts: 1834
Joined: Oct 24, 2010 15:16
Location: Ярославль
Contact:

Re: What happened to the Emscripten branch?

Post by VANYA »

The issues reported by VANYA seem to be gone
Yes now everything compiles and work without errors. I tried uploading images the same way, it also works fine. The mouse works, but GetMouse always returns 0, so there is no way to know when the mouse is outside the graphics window:

Code: Select all

Dim shared As Integer x, y, buttons, res 

type em_callback_func as sub
#ifdef __FB_JS__
   declare sub emscripten_set_main_loop cdecl alias "emscripten_set_main_loop" (byval func as em_callback_func, byval fps as integer = 0, byval simulate_infinite_loop as integer = 0)
#else
sub emscripten_set_main_loop(byval func as em_callback_func, byval fps as integer = 0, byval simulate_infinite_loop as integer = 0)
   if fps = 0 then
      fps = 60
   end if
   
   do
      func()
      sleep 1000/fps
   loop
end sub
#endif

' Set video mode and enter loop
ScreenRes 640, 480, 8


sub mainloop()
	
    ' Get mouse x, y and buttons. Discard wheel position.
    res = GetMouse (x, y, , buttons)
    Locate 1, 1
    If res <> 0 Then '' Failure

#ifdef __FB_DOS__
        Print "Mouse or mouse driver not available"
#else
        Print "Mouse not available or not on window"
#endif

    Else
        Print Using "Mouse position: ###:###  Buttons: "; x; y;
        If buttons And 1 Then Print "L";
        If buttons And 2 Then Print "R";
        If buttons And 4 Then Print "M";
        Print "   "
    End If
	
End Sub

emscripten_set_main_loop @mainloop
For VANYA, this is the file to replace
Thank you very much!

---------------

ASM inserts don't work or shouldn't they work?
angros47
Posts: 2321
Joined: Jun 21, 2005 19:04

Re: What happened to the Emscripten branch?

Post by angros47 »

You are welcome!
Asm inserts should work, in theory (clang should be able to handle inline assembly on its own), but I remind you that you cannot use x86 assembly, because the target is not an x86 processor (the compiled code is supposed to work on pc and phones in the same way)
angros47
Posts: 2321
Joined: Jun 21, 2005 19:04

Re: What happened to the Emscripten branch?

Post by angros47 »

Ok, found how to get the text mode to work, too

The file fb_shell.html contains a mistake, because it creates the terminal after the FB code is run, while it should create it before. To fix it, the lines:

Code: Select all

      var Module = {
		preRun: [],
        postRun: [function() {
			__fb_rtlib.console.open(80, 25);
		}],
must be changed to:

Code: Select all

      var Module = {
		preRun: [function() {
			__fb_rtlib.console.open(80, 25);
		}],
        postRun: [],
There are two files named fb_shell.html: one is located in the lib/ directory, the other is located in lib/freebasic/js-asmjs/ : they both need to be patched: the first one is used to create the second one, while the runtime is compiled. The second one is the one included in the final compiled program, so it's the one that needs to be patched (but if the first one is not patched as well, the first time the runtime is rebuilt the patch might be undone)
VANYA
Posts: 1834
Joined: Oct 24, 2010 15:16
Location: Ярославль
Contact:

Re: What happened to the Emscripten branch?

Post by VANYA »

angros47! Well done, the console is working. Isn't it time to make all the changes at least here: https://github.com/jayrm/fbc/tree/emscripten ?
After all, by and large the HTML compiler is already working. Another thing is the documentation, here you need at least a separate page explaining how applications work with an emphasis on events. Even the simplest animation should be in an emscripten loop and this should be reflected in the article. And of course you need to explain the installation FB+emscripten. I do not insist that you need to deal with the documentation, you have already done a lot. Maybe fxm ask? Maybe he won't refuse :) I would write it myself, but I'm afraid such an illiterate article from the point of view of English grammar, few people will understand :)
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: What happened to the Emscripten branch?

Post by coderJeff »

I've rebased and pushed changes to https://github.com/jayrm/fbc/tree/emscripten including the patches posted in this topic from angros47

I tested on windows x86_64 only. One issue and work-around (hack) is below.

Prerequisites
- git installed
- development environment for fbc, rtlib, & gfxlib2
- understand PATH environment variable
- comfortable using a command prompt
- build of latest fbc (that includes latest changes)

Install Emscripten
d:/emsdk.git is assumed for installation directory

Code: Select all

d:
cd \
git clone https://github.com/emscripten-core/emsdk.git emsdk.git
cd emsdk.git
emsdk install latest
emsdk activate latest
emsdk_env.bat
Build FBC libraries
d:/fb.git is assumed for latest fbc checked out from repository

Code: Select all

d:
cd \fb.git

REM whatever you use to set-up your fbc build environment (sets PATH)
call c:\batch\setpath.bat fbgit32

REM add-in the emscripten build environment (also sets PATH)
call d:\emsdk.git\emsdk_env.bat

REM build the rtlib and gfxlib for emscripten
make rtlib gfxlib2 TARGET=asmjs-unknown-emscripten ENABLE_STANDALONE=1
Hook for 'emcc.bat'
Here's the issue: fbc expects the supporting tools to be executable and emscripten font-end on windows uses 'emcc.bat' (plus python in the background) and fbc can't directly call a .BAT file. Here's a cheap hack to make it work:
Compile the following source and copy 'emcc.exe' to 'd:/fb.git/bin/js-asmjs/emcc.exe'

Code: Select all

'' emcc.exe to call emcc.bat and pass all arguments
''
'' tested, but not well tested...
''
function EscapeArg( byref arg as const string ) as string
	dim ret as string = """"

	for i as integer = 1 to len(arg)
		select case mid( arg, i, 1 )
		case """"
			ret &= """"""
		case else
			ret &= mid( arg, i, 1 )
		end select
	next

	ret &= """"

	function = ret

end function

dim cmd as string = "emcc.bat"

dim i as integer = 1
while( command(i) > "" )
	cmd += " " & EscapeArg( command(i) )
	i += 1
wend

'' assumes 'emcc.bat' is on PATH

var result = shell( cmd )

end result

Code: Select all

fbc emcc.bas
copy emcc.exe d:\fb.git\bin\js-asmjs\emcc.exe
Build an FBC program from fbc source
The first time this runs, emcc takes a long time because emscripten needs to build it's own runtime library and store the result. After the first time, compile times should be much quicker.

Code: Select all

REM whatever you use to set-up your fbc build environment (sets PATH)
call c:\batch\setpath.bat fbgit32

REM add-in the emscripten build environment (also sets PATH)
call d:\emsdk.git\emsdk_env.bat

REM build program.bas
fbc -target js-asmjs program.bas
Run 'program.html'
In the directory where 'program.html', 'program.js' & 'program.wasm' was created

Start a server:
> python -m http.server

Then, browse to the directory:
http://localhost:8000/

Or the program:
http://localhost:8000/program.html
angros47
Posts: 2321
Joined: Jun 21, 2005 19:04

Re: What happened to the Emscripten branch?

Post by angros47 »

Thank you!
VANYA
Posts: 1834
Joined: Oct 24, 2010 15:16
Location: Ярославль
Contact:

Re: What happened to the Emscripten branch?

Post by VANYA »

coderJeff , Thank you!

angros47, I unfortunately see that threads are not working (threadcreate).

when using threadcreate:
wasm-ld: error: unable to find library -lfbmt
wasm-ld: error: unable to find library -lfbgfxmt
I tried linking with the standard libfbmt.a, libfbgfxmt.a, but this produced an error:
wasm-ld: error: unknown file type: array_erase.o
Maybe you will have time to watch this?
angros47
Posts: 2321
Joined: Jun 21, 2005 19:04

Re: What happened to the Emscripten branch?

Post by angros47 »

The standard libfbmt.a and libfbgfxmt.a are compiled for x86 platform, not for emscripten, so they will never work. You would have to recompile the main library with threads active under emscripten.

When V1ctor developed the port of FreeBasic for Emscripten, Emscripten didn't support threads, so that part of FreeBasic has never been ported. I guess it can be ported now, but it is not something that needs to be fixes, it's something that needs to be implemented.
VANYA
Posts: 1834
Joined: Oct 24, 2010 15:16
Location: Ярославль
Contact:

Re: What happened to the Emscripten branch?

Post by VANYA »

angros47 wrote:The standard libfbmt.a and libfbgfxmt.a are compiled for x86 platform, not for emscripten, so they will never work. You would have to recompile the main library with threads active under emscripten.

When V1ctor developed the port of FreeBasic for Emscripten, Emscripten didn't support threads, so that part of FreeBasic has never been ported. I guess it can be ported now, but it is not something that needs to be fixes, it's something that needs to be implemented.
Clear, thanks for the clarification.
angros47
Posts: 2321
Joined: Jun 21, 2005 19:04

Re: What happened to the Emscripten branch?

Post by angros47 »

I just found out something interesting: V1ctor in the past stated, here viewtopic.php?p=212188#p212188 that any loop waiting for the user must be changed into a function called every x milliseconds. This means that programs featuring one main loop must be modified to run in Emscripten, or they will hang the browser.

Well, looks like in modern Emscripten that limit has been more or less overcomed: https://emscripten.org/docs/porting/asyncify.html https://emscripten.org/docs/porting/ems ... nment.html

Code: Select all

declare sub emscripten_sleep cdecl alias "emscripten_sleep"  (t as uinteger)

screenres 640,480   'That's needed, for now, because the text terminal mode has a bug that causes an exception

for i as integer=1 to 1000000
    print "Iteration ";i
    emscripten_sleep 1
next 
It must be compiled with:

Code: Select all

fbc -target js-asmjs test.bas -Wl "-s ASYNCIFY=1" 
and you can do loops. Perhaps the function emscripten_sleep should be called either by sleep or by screensync
VANYA
Posts: 1834
Joined: Oct 24, 2010 15:16
Location: Ярославль
Contact:

Re: What happened to the Emscripten branch?

Post by VANYA »

angros47 wrote:I just found out something interesting: V1ctor in the past stated, here viewtopic.php?p=212188#p212188 that any loop waiting for the user must be changed into a function called every x milliseconds. This means that programs featuring one main loop must be modified to run in Emscripten, or they will hang the browser.
Oops, that's what I needed! Now I was able to port one of my games to WEB. Thanks!

PC game (source code, game description and rules):

https://users.freebasic-portal.de/freeb ... clone.html

Online game , it works for me on firefox :) :

https://users.freebasic-portal.de/freeb ... clone.html

In the online version, there is no choice of the difficulty of the game due to freezes with the INPUT operator, but in principle it is already good. For the future, to me will have to write some kind of INPUT operator for web versions.
angros47
Posts: 2321
Joined: Jun 21, 2005 19:04

Re: What happened to the Emscripten branch?

Post by angros47 »

Tested it under Firefox, and it works for me, too, great work!

And actually, I was surprised when you told me you needed threads, because threads are almost pointless without loops, so loops were a priority. Now it's more clear.
angros47
Posts: 2321
Joined: Jun 21, 2005 19:04

Re: What happened to the Emscripten branch?

Post by angros47 »

Ok, I have found why, when text only mode is used, an exception is thrown at the end (that is annoying, since it prevented the use of asynchronous mode)

The issue is caused by the file io_getsize.c in the src/rtlib/js directory.

The original file features the function

Code: Select all

FBCALL void fb_ConsoleGetSize( int *cols, int *rows )
{
	int res = EM_ASM_INT({
        return __fb_rtlib.console.size_get();
    }, NULL);

	*cols = res & 0xFF;
	*rows = (res >> 8) & 0xFF;
}
that needs to be modified to:

Code: Select all

FBCALL void fb_ConsoleGetSize( int *cols, int *rows )
{
	int res = EM_ASM_INT({
        return __fb_rtlib.console.size_get();
    }, NULL);

	if( cols != NULL ) {
		*cols = res & 0xFF;
	}

	if( rows != NULL ) {
		*rows = (res >> 8) & 0xFF;
	}
}
Otherwise, the routine fb_DevScrnUpdateWidth will call it with the second parameter set to a null pointer, and that will be detected as heap corruption by the internal debugger
VANYA
Posts: 1834
Joined: Oct 24, 2010 15:16
Location: Ярославль
Contact:

Re: What happened to the Emscripten branch?

Post by VANYA »

angros47 wrote:Ok, I have found why, when text only mode is used, an exception is thrown at the end (that is annoying, since it prevented the use of asynchronous mode)
Yes, now the console seems to work fine. Good job!

angros47, is it possible to do such things by default (at the compiler level)?

1) compilation flags:
-target js-asmjs
-Wl "-s ASYNCIFY=1"

so that you can just compile normally: fbc file.bas . All the same, you cannot do without these flags in a real program.

2) replace sleep with emscripten_sleep command.

Is it possible to disable the console window in the final html file if, for example, the program works only in graphical mode?
Post Reply