Deno Standard GUI

[rsp]

Has there been any discussion about having a standard way of doing GUI in Deno? I don't necessarily mean that it all has to be in the core or std modules, and I don't mean anything complicated or very high-level, just something that lets us open native windows, show dialogs and generally create GUI apps using just JS/TS.

In issue #1629 about WebGL from January (the only issue related to GUI that I've found) @ry you said that you want to support WebGL in core eventually. If it is possible to open a native window and draw using WebGL then it will be possible to use it to create a GUI framework as a third party module, but thinking about it I started wondering about a Web way to do GUIs, which is really HTML. I'm not arguing that it is the best way but this is certainly the most Web-style way. And since one of the goals of Deno is to have a similar API for the things that people are used to in browsers, then it seems like something worth thinking about.

If Deno doesn't use a Web way of creating GUIs then soon it will have to reinvent HTML and CSS, plus some logic of dispatching the click events etc. and it will soon get quite complicated.

If, on the other hand, Deno is able to open a window with HTML even with very limited functionality (like only text and canvas, for example), then it will be relatively easier to not go too far away from what people are used to in the browsers.

Of course having something like Electron bundled in Deno might be a little overkill (or maybe not? any thoughts on that?), but having a possibility to write an Electron-like framework as a third-party module without producing a separate Deno binary to be able to do that, would be something to think about.

Running deno run https://www.example.com/gui-example.ts to start a GUI program, instead of deno-electron-like-fork run https://www.example.com/gui-example.ts would be something to consider.

I wonder if there has already been any discussion about it.

[kitsonk]

@ry and I recently talked about it, and we presented the intent at TSConf, but there was no issue for it. There was some conversation on Gitter recently as well.

[amirbit711]

very good

[sholladay]

An Electron-ish thing built with Deno would be a big deal to me. I'd put a lot of effort into switching my apps to it, since the security improvements alone would be quite substantial. I suspect the resource efficiency would be much better, too, if Deno's startup time is anything to go by.

[cztomsik]

from scratch, I wanted to use at least webrender initially but I'm targetting raspberry pi and that only has opengl 2.1

[martin-braun]

Also checkout https://github.com/webview/webview_deno

[anthonytietjen]

Check out https://webui.me/

[DougAnderson444]

Check out https://webui.me/

@anthonytietjen Yep that's already down below #3234 (comment)

[Cassielxd]

This is my communication case using the combination of deno and tauri
https://github.com/Cassielxd/tauri-cc

[rsp]

@kitsonk Thanks, I didn't know about it. Having a full-blown browser bundled would be something absolutely perfect, if only the overhead is not prohibitively expensive - but since you are already considering it with Ryan then I guess that happily it isn't.

Will what you were presenting with Ryan at TSConf about this be available online to watch? Or maybe it is already?

It would be great if Servo could be used here. The tight coupling with SpiderMonkey seems like a problem, unless it would be used just as a completely separated frontend and communicate with the main Deno process with a WebSocket or something like that, maybe having Deno transpiling the frontend code from TS to JS on the fly.

It is actually pretty interesting how it could be done - from GUI windows being just lightweight browsers, getting all assets via some proxy or file protocol and communicating via WebSockets with the Deno process, to having some duality like Electron, to having a browser part using the same engine or even the same process and event loop as the main Deno code. I wonder what is your and @ry's idea of what would be optimal here.

[Cassielxd]

This is my communication case using the combination of deno and tauri
https://github.com/Cassielxd/tauri-cc

[rsp]

@sholladay Yeah, I am currently writing an app that needs a GUI which I am doing it in Electron but it is something that could be written in Deno with the current API with no problems if I could only do GUI with Deno (and unfortunately it cannot be a web app). I am concerned exactly with security of what I am doing right now, that the app will have full access to user's data, which could be nicely restricted with Deno. Not to mention the transpilation steps etc.

[brainkim]

Having a built-in module which bundles a cross-platform webview implementation would be the killer-feature that would get me to start experimenting/migrating from node.js.

[jardicc]

I have some basic feature requests. Few native dialogs. Alert, prompt, confirm and file/folder picker/save dialogs. Because sometimes you have app running in the background it will throw an error but user can't notice it so you have to show some dialog about what is going on and in Node.js there is no easy way to do that.

[erlend-sh]

Ya’ll should have a chat with @nothingismagick and the other fine folks at Tauri: https://github.com/tauri-apps/tauri/

A lot of what has been described here, they’re already working on or have already thought a great deal about.

[Cassielxd]

This is my communication case using the combination of deno and tauri
https://github.com/Cassielxd/tauri-cc

[rsp]

@erlend-sh Are you sure this project is alive? The tauri.studio ___domain (from github.com/tauri-apps/tauri.studio) redirects to a ___domain parking page and on tauri-apps.org (the documentation link in the readme of github.com/tauri-apps/tauri) I get ERR_CONNECTION_TIMED_OUT:

This site can’t be reached tauri-apps.org took too long to respond.

[nothingismagick]

Yes - its alive. We're preparing our site to launch this month.

[jerrygreen]

Unfortunately Deno support is deep under the plans, not in 2021, and who knows when

[nothingismagick]

@rsp - we've had so many moving parts that we couldn't justify publishing the docs about how to use tauri until we've got a stabilized API and build system. Feel free to drop by our discord server if you want some quick support about getting setup with it.

On a sidenote, we do plan to release a deno CLI and rust integration once deno is stable enough for that too.

[rsp]

@nothingismagick Thanks for the info. The project after reading the readme looks almost too good to be true. Really looking forward to reading the docs when they are available. It's great to hear that you plan releasing a Deno CLI as well.

[nothingismagick]

yeah - we just a comment from a svelte user today. they confirmed a final linux build size of ca 600KB.

[nothingismagick]

But I know we can break the 500KB boundary for sure. Here's the discord if you want to stop in and have a chat:
https://discord.gg/SpmNs4S

[kitsonk]

It would be great if we remembered that this is a Deno issue tracker. 😄

[adam-cyclones]

I would very much like to port my desktop environment to Deno, watching.

[hansoksendahl]

We need a deno powered electron like experience. It's the killer app. Please someone I'm not a system programmer. 🤷

I will sponsor anybody that is working on this. For real hansoksendahlₐₜgmail.com

[felipetesc]

This project it is NOT mine, perhaps it can help some people: https://github.com/marc2332/tauri-deno-starter

[JasperVanEsveld]

Hey everyone,

I think I have something that fits this perfectly, it is basically Deno + IPC + Webview.
If you just want to check it out you can find it here.
It currently only works on windows.
There is an example that I tried to keep as simple as possible, if something isn't clear just post an issue.

It works using a modified Deno binary to compile your programs, just as deno compile would do. Instead of starting deno like normal it also launches a webview window that can communicate with deno.

This is more of a proof of concept, if added to deno I would see it as follows:
With an additional flag for compile, for example compile --webview.
When compiled with --webview the resulting binary will start webview on launch with some form of IPC between them.

The main issue would be windows, which by default opens the command line.
Unfortunately I don't see any other solution than to maintain an extra binary that has the line #![windows_subsystem = "windows"] in main.rs. This binary is than used as the source when compile --webview is used

[justinmchase]

Very cool, this is looking much closer to what probably needs to be the kernel of the solution.

Questions

Because its using compile is it resolving all imports and packaging up all js/ts files? Does it also package in the html file from the --webview-url? Will it only package up a single html file? Or how would you deal with relative path assets, such as css and images, runtime dynamic imports, etc.?

Once the webview is loaded, if you're using native imports how will it resolve relative path imports? It has hooked into the browser to resolve embedded relative path imports?

How does it handle ts/tsx imports in the webview? Are you using the browser native module imports? Or are you injecting Deno into the render process also and allowing it to do the importing?

Comments

It may be interesting to look at this repo https://github.com/ayame113/ts-serve, which is using an interesting technique to make deno modules available in the browser by essentially down-compiling ts files into javascript but preserving the file extension and swapping the content-type header to application/javascript, which lets the browser load the file without requiring the imports themselves to be altered. Since TS is a super-set of JS it works.

It seems like it would be pretty useful to discuss some ideal specs for such a tool. I think some things were learned from Electron but the way it ended up isn't actually ideal. Beyond not supporting ESM, electron has some other aspects which seem to have evolved in a bad direction.

Things to avoid

• synchronous initialization process
• reliance on complex heavy-weight bundling technologies
• prohibitively secure by default

Things that are good

• Main process and render process(es)
• Asynchronous message passing between render and main processes
• Main process is deno, Render process is just a browser
• Simple by default, easy to bootstrap
• OS agnostic apis for common desktop functions
• All in a single tool, preferrably deuteron has all that is needed
  • tightly coordinated with deno, doesnt fall way behind latest
  • possibly just part of deno itself
  • no reliance on mutliple, hacky, complicated tools and bundlers

It would be interesting to hear others peoples actual specifications for such a tool.

[justinmchase]

In case anyone hasn't seen this already, I highly recommend going out to this thread and reading some of the comments, such as this recent comment:
electron/electron#21457 (comment)

Also, this one is the outline of all the major issues in Electron which have been holding it back, and we should avoid reproducing:
electron/electron#21457 (comment)

Its also worth noting that this issue was originally filed in Dec, 2019 and it seems to have not been addressed at all and the situation there is very frustrating. In 2017 I could go and make an electron app in under an hour and it was a real pleasure and now suddenly, due to their failure to adjust the framework around ESM it has become a pretty horrible experience. There is absolutely an opportunity here.

[JasperVanEsveld]

Thanks for the well structure response!
I wanted to get the ball rolling on, what I think, would be a good approach.
So if anyone else has thoughts on either the approach or what a final spec would look like, feel free to comment!

I will try to adres your questions/comments.
Essentialy deuteron is just Deno + WRY, so most answers will be related to either of them.

Questions

Packaging

Nothing from the frontend is packaged, the --webview-url is resolved using resolve_url_or_path and passed to WRY.
This will load whatever html file it points too, any further imports the html file does are fetched as a normal browser would.
So if it is a local file it would be the same as opening it in the browser.

Now as to what would be good for a final product, that is a different question.
Packing everything would make the resulting binary portable, which would be great.
However, as you noted yourself, I don't think it is trivial with css and images.
Resolving all dependencies is very tricky.

One solution that maybe more feasible would be to add a --resources <file|dir> flag to deno compile.
Just to provide a (zip)file that deno can unpack and serve, so webiew can load --webview-url localhost:1234.

A --resources flag might be useful for other applications as well, for example to include json data. Maybe it has been mentioned before but I couldn't find an issue.

Other questions

Once the webview is loaded, if you're using native imports how will it resolve relative path imports?

The initial --webview-url is resolved using resolve_url_or_path (same as deno uses for the entrypoint of your code) this resolves a relative path to an absolute path.
From there webview resolves just how a browser would normally do it.

It has hooked into the browser to resolve embedded relative path imports?

No the only connection between the two is simple string message passing.

How does it handle ts/tsx imports in the webview?

Not (see next section for some ideas to solve that)

Are you using the browser native module imports? Or are you injecting Deno into the render process also and allowing it to do the importing?

Just using the native browser imports

Comments

First of I did not know of ts-serve, this sounds amazing and I will probably use it in any future projects.

I like the idea of supporting TS, as Deno supports it already making the frontend JS only would be jarring.

These are some options I could think of:

Transpile during compile

Transpile and bundle the TS files at compile time and include them in the executable. In my opinion this is not feasible, maybe some of the bundling methods of Deno can be reused but this just sounds prone to bugs. And as mentioned before bundling the frontend might not be trivial.

Vanilla

This is what deuteron currently does, change almost nothing to the webview part. It is just a browser with IPC and some window control functions (minimize, maximize, etc..).
Making it easy to maintain.

If you want TS you can load a page served by deno that is using something like ts-serve.
So it can still be done, but requires more setup work and can not be used for external URLs.

Service Worker

I am not 100% sure on this, as I have never used service workers. But I think it could be possible to intercept any TS request with a service worker and use Deno to transpile it. You could even implement some form of caching.

While more work compared to the vanilla approach it is still managable to maintain, without breaking any normal browser functionality.

It is actually pretty similar to ts-serve, transpiling on the fly, but working out of the box. Personaly this sounds like the best option.

[JasperVanEsveld]

Just an FYI, hadn't really tested this yet
If --webview-url points to a local html file you can't import JS modules, as this trips CORS just like opening that same file in the browser would.
So for now serving your frontend files with Deno and then using --webview-url localhost:port would be required to use JS modules

[JasperVanEsveld]

To expand on the basic example I adapted a library I made a few months ago.
The library gets a download link from a youtube id/url.

The desktop app uses github pages to load the frontend, making for a nice one line compile:

deuteron --title yt-download --webview-url "https://jaspervanesveld.github.io/yt_download/" --allow-net https://deno.land/x/[email protected]/mod.ts

I really hope something like this can be added to Deno as it has a lot of potential.
This was thrown together in a few hours, most of which was spend on frontend stuff.

[justinmchase]

I really hope something like this can be added to Deno as it has a lot of potential.

By "this" you mean all the features of deuteron? I agree that would ultimately be the right place, because with deno most people are going to be doing "deno upgrade" and if deuteron falls behind the main version too far then its going to break that model where they need multiple versions locally, and to be constantly worrying about having the right versions and compatibility, etc. If it was natively supported it would be much more likely to always be in sync and to have a more rapid adoption.

It would be really cool to break down the individual features that would be required and see if they can't get added incrementally. I think you have at least 3 that I heard:

• deno binary with webview
• support embedding resource files at compile time (non-import, non-code files)
• launch with --webview-url

Maybe those can be broken down further? I think the deno team has been pretty gracious about accepting reasonably scoped PR's from outside teams. If it was broken down into small pr's, the sum of which added to this end result it could work.

Local File Strategy

So given that the local file webview requires a server, and it sort of seems like a given to me that many apps will want to have stand alone binaries which have all code/resource files bundled with it (not pulled from a url), it seems possibly like having an embedded webserver be part of the webview launch.... What would that look like?

sequenceDiagram
    Main->>+Server: launch 127.0.0.1:12345
    Main->>Render: webview launches
    Render-->>Server: http://127.0.0.1:12345/index.html
    Server->>Main: file("./index.html")
    Server->>Render: index.html
    Render-->>Server: http://127.0.0.1:12345/main.ts
    Server->>Main: downcompile("./main.ts")
    Server->>Render: main.ts, application/javascript

Loading

Could that be driven from a runtime module?

// main.ts
import { Application } from "https://deno.land/x/deuteron/mod.ts";

const app = new Application({
  webviewUrl: "./index.html"
});

// todo: ipc handling here...
await app.run();

This would sort of push the launching of the webview into a deno runtime api instead of a command line startup flag, such as maybe

const window = Deno.webview(url);

[JasperVanEsveld]

@justinmchase
Made 2 issues for the things that are nice to have for other applications as well.
Also opened a PR for the --no-terminal flag, as that was fairly easy to do.

Do you know how I can get more attention on them, or should I just give it some time?
Haven't contributed to Deno before and don't want to needlessly ping people
Issues: #17990 #17994

[aapoalas]

Especially if stuff is easy to do, then the absolute easiest thing to do is to write a PR: I'll be happy to review and approve (when I have time) if the PR looks fine and a test is included :)

[JasperVanEsveld]

That would be great!
The PR for --no-terminal is already up: #17991

As I said before I haven't contributed to Deno before, so please tell me when something is missing/wrong.
Also I haven't added any tests, though it only changes 2 bits of the executable
If you know what would be a good way to test it, then I will be happy to add them :)

Issue #17994 was a bit more complicated and I am also not 100% sure on what approach to take
So haven't started on a PR for that

[justinmchase]

I've been conversing in this chat hoping some others from this fairly large discussion thread would chime in but probably it would make sense to drive this forward into different discussions. Some would make sense on the project repo (imlementation deails) but it may make sense to start a new discussion here in this repo that is more of a specific proposal. I would probably recommend getting buy-in from the deno code owners before going too far because if you hit a road block with them, then it could be a lot of effort for nothing.

So perhaps make a new discussion in this repo, and instead of discussing the possibility of a standard gui propose a more specific solution.

• High level description and perceived benefits
• Outline what changes would be needed to the deno runtime
  • Lists costs, file size impact, startup performance, etc.
  • List solution options
• Cite specific technology choices (such as webview) and their implications
• Give a high level architecture and initial feature spec
• Give roadmap for cross platform support
• Describe the architecture
  • Multi-process
  • ipc
  • special compile, etc.
• List unknowns or areas of challenge without a known solution

I feel like if you propose it as a solution and the team has a clear vision of how it could be progressed and the costs / benefits, you could get buy in and go from there.

[JasperVanEsveld]

Sounds good, I will look into it in the coming days (when I have time)
Thanks!

[felipetesc]

Just another idea: I found this project: https://github.com/oooutlk/tcltk - Rust bindings to Tcl/Tk commands

[esibife]

It was interesting

[seriosi1]

It was interesting content

[nicosql]

@rsp have you seen this? It is written in pure-C, works like a dream.

https://github.com/webui-dev/deno-webui.git

[AlbertShown]

The latest Deno webui wrapper seems stable now.

[tvortsa]

The latest Deno webui wrapper seems stable now.

Did you send any image from backend to front by webui deno ? I did try everifyng and still cant do that :(
Even in version 2.5.4, but its not official relise, i still hope that in oficial release all be work for me.

[AlbertShown]

Did you send any image from backend to front by webui deno ?

The binary send/receive gets updated and fixed in the C library, but not yet updated in the Deno wrapper.

[tvortsa]

Did you send any image from backend to front by webui deno ?

The binary send/receive gets updated and fixed in the C library, but not yet updated in the Deno wrapper.

...and this means that it is not suitable for Deno GUI desktop!

[AlbertShown]

Yep, without Deno wrapper update it's not suitable for Deno GUI desktop

[rivy]

Did you get the webui binary to unzip? I got an error. (https://raw.githubusercontent.com/webui-dev/webui/main/Release/webui-windows-x64-v2.3.0.zip)

On the plus side I was also able to hack the Rust version together so that it now compiles (Deno is written in Rust, so that might become handy too)

It's actually a TAR file, not ZIP. Use 7z; it will open it correctly. See webui-dev/webui#120.

[jingkaimori]

with current FFI interface, we can use sdl2 to provide native gui for desktop application:
https://deno.land/x/[email protected]

[justinmchase]

This is actually really cool. Could make indie games with it for sure.

[7flash]

can i bundle deno runtime to execute scripts from my electron app?

[justinmchase]

You certainly could do that in a child process but I'd be pretty surprised if you could have it run in-proc with node, I'd have to assume the two instances of v8 couldn't share threads but I'm not really sure.

But if it was out of proc then it would work but would be fairly complicated, wouldn't be able to use it for UI and you have to have an extra communication channel to hop from render to main to child process.

[hansSchall]

I noticed that Deno already supports JSX/TSX, so maybe we could take advantage of that. By using a concept similar to ReactNative a full GUI-API could be built without the overhead of a browser instance. The implementation basically consists of React/Preact (Preact already works with Deno/Fresh), Yoga-Layout (C/C++ implementation of modern CSS Layout) and bindings to platform-native ui components (already existing for Windows, Mac (ReactNative Desktop), Android and iOS (ReactNative)). This solution would preserve the speed of Deno and could be (if it's not too big) included by default (providing an alternative version without GUI support for server-side applications like Deno Deploy). The API should be designed in a way which allows code to be shared with Fresh. Although the discussion focusses on desktop, this paves the way for mobile development.

Those who wish to use HTML/CSS could use a Webview/Iframe (Chromium loaded on-demand; automatically downloaded if needed and not already installed (on Windows the Edge installation could be used)).

[martin-braun]

I like JSX/TSX, but please let's not bind ourself to React. Just wanting to point this out, not saying that you implied this.

[nicosql]

It is maybe 30 minutes of work to setup a project that offers all of this functionality soup @hansSchall is mentioning, so regardless of the intention or not I don't understand request like these.

Deno is doing very good things like KV / watch / cron / docs etc.. THOSE are the tools I want them to keep building - we don't need them writing simple glue code for us regardless of the framework.

[7flash]

Here I made a demo compilable deno server runs frontend in a chrome window: https://github.com/galaxydo/deno-chrome-app

[jardicc]

When you propose to run Deno UI in webview or something like that. It does not work great when you e.g. need only alert/confirm/prompt dialogs... it also does not include tray icon access... also custom dialogs like folder picker, file picker would be great. Being able to show these without rendering webpage first would be cool. Also webview has usually problem with multi-monitor setup.

I personally used NodeJS to create plugin for Photoshop because those plugins are in JS... so makes sense I can share code between external and internal proces. So the use case is not always server only.

[BoltDoggy]

https://github.com/bramblex/niva

bramblex/niva#17 这里有关 Nodejs 调用 Niva 的示例。

niva 基于 Tauri wry 使用 rust 实现，和 deno 结合应该会更加友好。