Opral
Accelerating lix by years: Inlang directories as lix repositories #2975
Replies: 6 comments 4 replies
-
|
+1 Getting rid of Git also means the possibility of developing new workflow concepts for non-developers, which we need to do at some point anyways. |
Beta Was this translation helpful? Give feedback.
-
|
agree. github features like pull requests, ci/cd, issues, etc. (that we need to rebuild to work locally + for non devs) are a distraction rn. PS we still need push, pull under the hood. you probably refer to abstracting it. |
Beta Was this translation helpful? Give feedback.
-
|
Yes, we have to push under the hood, but we can abstract it and define the workflows of how this should be done. |
Beta Was this translation helpful? Give feedback.
-
|
+1 The last months showed that if we could take a shortcut with Lix, we should. The only thing that counts in the next months is the number of differentiating features we can integrate to overtake the competition. Nobody cares if we can do that in theory. UX Tech Incremental ability
-> At this point, we can outcompete the competition and start monetization |
Beta Was this translation helpful? Give feedback.
-
|
haha you are too early with zipped files. depends on the research @janfjohannes and I find out. long term zipped files are the best thing we can have for interop and portability (computers and apps that all expect a "file" rather than a directory). |
Beta Was this translation helpful? Give feedback.
-
|
+1
A small drawback would be that we lose code lint rules for non dev apps (only dev apps have the code), but we can still save the results in the project. This also makes the case that some inlang SDK will only be available in dev apps (code lint rules, code context). Two things will be crucial:
Time is well invested in testing this with a prototype to figure out the crucial parts and reveal possible blind spots. After that we should decide what we need to build in the next few months. |
Beta Was this translation helpful? Give feedback.
-
|
@NilsJacobsen @samuelstroschein @NiklasBuchfink (btw. mentions don't work if you do the 's in @janfjohanne's)
we all agree that we want something in this direction in the long run, but what is doable in the short term would be quite a bit more modest. this conversation reads sometimes a bit like discussing how we can remove all roads and thinking about how cool car parks for flying cars would be when the elephant in the room is that no one can build a flying car that is usable in everyday life yet and its not even certain this is possible at all. what IS a viable simplification that is mentioned here:
|
Beta Was this translation helpful? Give feedback.
-
|
i also want to add a bit more detail to one example of problems, also because this illustrates that the issues to solve are not just technical but logical issues. imagine you have a repo and a subrepo. the parent repo has 2 branches and the subrepo is in the parent brnach at "/sub". How would you map the parent branches to the subrepo branches? all current approaches allow only mapping one branch from the subrepo to all branches in the parent repo, the subrepo can have its own independent branches, but only if they are merged to the linked branch will they be available in the parent repo. this is probably not what we could use as this would mean you could not open a test branch created in the parent repo when you would open an inlang project in fink. so now lets say we build tooling that would magically create and link the parent repos branches to the subrepos branches, problem solved? now it would be impossible to use the subrepo without the parent repo which was the main reason we introduced subrepos. what happens when you merge the branches in the subrepo? what happens if you merge the branches in the parent repo but our tooling is not running? what happens if history of the parent branch is rewritten? how wold the parent repo even see that changes in the subrepo happened? what happens if our tooling is not running propperly while changes to the repos are made? |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the insights. I worked on a new proposal over the weekend after it became clear on Friday that we need 0 git compabtility. All concerns and technical complexity stems from git compatibility. Users don't care about git. |
Beta Was this translation helpful? Give feedback.
-
|
A follow-up on this proposal will follow shortly. Freeze discussions in the meantime. |
Beta Was this translation helpful? Give feedback.
-
LOOM
https://www.loom.com/share/1cddb195df2f41ed98c320fab6001b33?sid=ea253213-e96f-4b89-a8e3-faf99738c4ea
TL;DR
If we find a way to “store” lix repositories in git that push and pull from a lix hosting service, we no longer have to host inlang directories on GitHub. If we don’t have to host inlang directories on GitHub, we will control the entire lix workflow (push, pull, commit, merge, ci/cd, etc) and don’t have to hack around Git and GitHub anymore.
Context
Inlang directories depend on external git repositories, which means:
We spent months building lix with little return in getting closer to differentiable lix features (the hosting platform, semantic merging, or third-party lix apps).
Scalability requirements are high because inlang apps operate on an entire git repo instead of just the inlang directory.
Proposal
Turn inlang directories into self-contained repositories to become independent of git.
Speed up the path to differentable lix features lix by years. Self-contained inlang repositories have lower requirements for hosting, network capabilities, security, and other aspects than touching source code.
Eliminate bugs caused by the lack of a repository [paraglide-js#44, cli#5, paraglide–js#51])
Benefits
Scalability requirements are much lower
Hosting inlang “files as repositories” leads to substantially smaller repositories, reducing the scalability requirements for networking, querying, and performance.
We don’t need to support 100MB monorepos from day one. An inlang “file as repository” is <1MB large.
Lix SDK controls push, pull, commit, etc.
Inlang apps, powered by the lix SDK, gain full control over change control workflows like merging, push, pull, etc.
We unblock the path for the lix SDK to start building the semantic merge API, auth, permissions, ci/cd, etc., because we do not have to consider the parent git repository anymore. Commit, push, and pull happens via inlang apps, not the git CLI.
Ideal workflow
Web app perspective (Parrot, Fink)
Dev app perspective (Sherlock, Paraglide)
Next steps
-- [ ] submodules?
-- [ ] subtrees?
-- [ ] something else?
Additional information
Things we invested time into that would be redundant
Lix lazy checkout (6 months effort)**
Inlang files (& other first lix use cases) are small enough and do not need lazy checkout. We could have invested 6 months of 1 full-time lix person on the lix hosting platform.
Fink GitHub auth, pull requests, rate limits shenanigans (recurring effort ≈2-4 months)
Fink is crippled by GitHub. Constant problems with auth, pull requests, and lately even rate limits (wtf). If we had the hosting platform, we could have invested the recurring work and pain of dealing with GitHub auth and pull requests into lix auth and change proposals.
Ninja (2 months effort)**
If we had the hosting platform, would we have built Ninja? Probably not. We would have invested the time into lix automation pipelines (CI/CD).
Inlang slot file storage (3-5 months effort)
We decided to ignore lix while designing the inlang storage format because:
If we wouldn’t need to hack around git, we likely would have decided to invest the time into the lix file extension API instead of inventing a slot file storage to work around git. 3-5 months of work that could have been invested into a generalizable lix solution instead of a custom inlang workaround.
Beta Was this translation helpful? Give feedback.
All reactions