We are going to be putting a lot more effort into making SN's local dev environment reliable and easy to use so it isn't a hurdle to contribution awards. It currently relies on a docker-compose system but nearly everyone experiences issues getting that setup. Someone will fix it until it "works for me" but then it breaks for other people.
Before I overhaul our setup, I wanted to see if stackers had any remarkable experiences to share that I could learn from. It can be a job you had where they did a great job at this, a project you worked on that did a terrible job, something you've made for yourself, or anywhere in between. What did they get right? What did they get wrong? What do you always like to see? Let it rip.
Once upon a time @Michelson_Morley shared this repo along with its contribution guide as an example of a stellar "getting started" experience. So if you have any resources whether a blog, book, or link to a repo where someone also did a great job, please share that as well.
For fun, every answer that helps me learn more about local project dev in service of us improving ours will get 1k sats.
1,000 sats paid 12 times
k00b's bounties
1597 sats \ 4 replies \ @ch0k1 6 Mar
This article tries to summarize all the options, their pros and cons and the best practices.
reply
Do you (or does anyone) have any recs for expanding out this container-based strategy? I know so little about this and it seems like it would be high value to understand it. But I don't know an obvious place to go to accomplish that goal.
reply
140 sats \ 2 replies \ @k00b OP 6 Mar
Was there anything in particular in that article that you liked?
reply
154 sats \ 1 reply \ @ch0k1 6 Mar
Listing all the three options you have (and outlining their pros and cons) in terms of setting up a common local environment:
  1. Using your machine as a host and installing everything on it (not recommended);
  2. Using virtual machines with Vagrant
  3. Using Containers with Docker
reply
77 sats \ 0 replies \ @ch0k1 6 Mar
In my experience, the Docker containerization is the best and most lightweight solution for running a predefined environment locally.
Other option (more expensive) might be to use cloud services as an remote development environment using terraforms for their quick setup and orchestration.
reply
Just to mention that kieran on Nostr has been having some success earning by broadcasting live coding to zap.stream. He codes the Nostr client snort.social and I think he built Zap.stream too and NostrNests.com. In case you hadn't considered it, you could maybe stream some live coding while you build out this site at minimum - or umbrella full on a coding Stacker.news(TV) station at maximum.
reply
117 sats \ 0 replies \ @k00b OP 6 Mar
Not a bad idea. It would probably help people familiarize themselves.
reply
Maybe I am biased, but I didn't find SN's setup to be bad. Docker Compose does a good job at the multi-process architecture that SN has.
My only gripes (that I can remember):
  1. I ended up carrying a diff like this in my working tree at all times to point the compose file at my own .env instead of the sample .env. I chose this option so I could modify .env without risk of committing it with secrets. Maybe there's some support for implicit dev .env file overrides so I don't have to carry this diff?
diff --git a/docker-compose.yml b/docker-compose.yml index 43cb702..d518be9 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -9,7 +9,7 @@ services: ports: - "5431:5432" env_file: - - ./.env.sample + - ./.env volumes: - db:/var/lib/postgresql/data app: @@ -24,7 +24,7 @@ services: depends_on: - db env_file: - - ./.env.sample + - ./.env ports: - "3000:3000" volumes: @@ -41,7 +41,7 @@ services: app: condition: service_healthy env_file: - - ./.env.sample + - ./.env ports: - "8080:8080" volumes: @@ -63,7 +63,7 @@ services: retries: 3 restart: always env_file: - - ./.env.sample + - ./.env expose: - "8080" ports:
  1. Not having a substantial database seed was challenging at times. IIRC, I made a couple of small seed file updates to help bootstrap a few use cases, but nothing substantial. But for many features, having a nice seed makes developing/testing easier. I think I've heard you mention working on this, so maybe it's improved since I last checked!
If I think of more, I'll add a follow-up comment.
reply
100 sats \ 1 reply \ @jp305 6 Mar
I agree, that is something that came up for me too... I would have the compose file link a .env.local file and git ignore this.
reply
110 sats \ 0 replies \ @k00b OP 6 Mar
Yep, I'll add that to the list of things to fix
reply
42 sats \ 1 reply \ @k00b OP 6 Mar
Not having a substantial database seed was challenging at times.
I built a seed a few weeks ago but haven't incorporated it yet. Good reminder!
reply
Awesome!
reply
What's a database seed?
reply
It's basically an initial set of data populated into a database.
The source code repo contains the database schema itself, so when you stand it up locally, the structure (all the tables) are in place. But, for the most part, they're empty tables. So when you open up SN locally, there's minimal posts, comments, users, etc.
A database seed would populate test data into the database, making it easier to develop or test certain features because you have test/sample data to work with.
reply
1152 sats \ 1 reply \ @chris42 6 Mar
My setup for each project:
  • debootstrap --variant=minbase to create a minimal debian base os in a directory
  • systemd-nspawn to chroot into debian base
  • A bash init script to install stuff, apt install. Put script in git
Pros: lightweight, I run full gui apps/web servers/dbs in there, everything is in a folder, can just delete it all and reinstall easily, no docker/kubernetes complexity
Cons: systemd-nspawn so no Mac support which is like 80% of Devs
reply
I've been using Linux (specially Debian) for a long time and never heard of debootstrap and systemd-nspawn before. By the Pros you listed it looks like something I'd love to incorporate in my toolbox.
Do you mind sharing an example repository with configurations and scripts you use só I can take a look on how you manage things?
reply
1041 sats \ 3 replies \ @BlueSlime 6 Mar
I like docker compose but I think it can become a crutch that props up overcomplicated setups.
I use docker with a design pattern called workbench. Everything runs inside one container, and it acts more like a VM that you can customize then distribute with your project.
When I want to setup my environment, I just launch the one container. If I have to make changes, I can change the source files then reload the container. It's also easy to login and get a shell prompt.
reply
I’ve been considering doing it all in one. Nice to hear from someone doing it that way. Docker’s documentation claims that’s the wrong way but it seems more straightforward do a dev env at least.
reply
I think the Docker devs are conflicted on this subject.
On one hand, they have adopted compose into their api, and want you to use it to build out micro services, which leans towards an infra-as-code model.
But otoh, they have also adopted a project called tiny into their container api, which offers better handling of multi-process containers.
If you want to go hard core, there's ways you can run debian systemd or alpine's manager within docker.
I try to keep my process count minimal so a process manager is overkill for me. I just run with tiny enabled and use tmux for basic management.
I run a webserver, postgres db and bitcoin node in the same container all the time, with tor and ngrok tunnels. Works flawlessly.
reply
reply
1030 sats \ 1 reply \ @jp305 6 Mar
I don't have much to share, but I just spent a couple hours trying to make it works and I find the compose setup to be great honestly. While I had some issues (CSP, network name) those are just small, resolvable annoyances.
Setting up the lnbits and polar stuff should be less manual, but I understand they are optional. Since they're both docker based, should be easy to have just another compose file which we can include?
reply
Setting up the lnbits and polar stuff should be less manual, but I understand they are optional. Since they're both docker based, should be easy to have just another compose file which we can include?
I agree, it would be cool if we could just opt-in to them via a config file instead of doing it manually.
reply
I think another alternative to consider would be to have some of the shared services deployed in a network-accessible location (like a dev/test deploy of SN) where developers could point their local NextJS app to, to develop against. I guess it would mostly be hosting a database instance and LND node, while the developers ran just the JS app locally. However, in this case, they'd be limited to which kind of changes they could make (UI and API only, no DB changes required). But, this might be nice for a new developer just getting their feet wet. It also lowers the hardware requirements for running the entire stack locally.
reply
1010 sats \ 0 replies \ @m00ninite 6 Mar
I've been meaning to learn nix. It seems like it would be great for this. I can't recommend it since I haven't used it extensively, but it may be worth looking into. If you want reproducible builds (as in, identical binaries each time you build), it's supposed to be great for that.
I'm a fan of docker-compose, but somehow it fails the "it just works" test every time I use it for dev env purposes
reply
1000 sats \ 0 replies \ @felipe 6 Mar
I like simplicity.
I never liked using Docker in production but I used to think it was ok for development but with time I started to notice that the complexity I was hiding when configuring a dev environment was coming back to bite me later down the road.
Today, I prefer having all things running locally and I take the time to understand each part of the system and how they fit together. If something is too heavy or messy or something to run on my workstation, I install it on a local server or on a VM on QEMU or VirtualBox.
Also, when I have the choice, I don't use complex build & deploy systems and magical tools anymore. I write my scripts by hand using plain old bash.
Basically, I went back to the way I used to do things before 2014, the year I met Docker 😅
Simplicity is prerequisite for reliability. - Edsger W. Dijkstra
:)
reply
Docker compose is a good way for managing local dev environment. I like this article Docker Compose Best Practices. You also need some scripts for initial setup and migrations (db schema, restructure...) For configuring project I use direnv which abstracts away environment differences.
I am also experimenting with Nix shell as development environment which gives you fast and lightweight development environment.
reply
1000 sats \ 0 replies \ @sebastix 6 Mar
Docker with docker compose with self-build Docker images From time to time I look into the latest updates from https://ddev.com/ if there any neat features I can copy into my own stack ;-)
reply
Googling around I found this great article by GH's VP of Eng. Has anyone used something like codespaces or replit for a multi-process/port application? I'm thinking it might be the easiest way to get someone onboarded quickly even if it isn't perfect in some ways.
Edit: the website hosting the blog also looks like a great resource.
reply
10 sats \ 0 replies \ @ch0k1 6 Mar
Well explained theory but unfortunately lacking particular code or setup examples
reply
20 sats \ 0 replies \ @Fabs 6 Mar
You can pay bounties multiple times in the same post? Awesome.
reply
Hey, I am not a Cody or devy. But I honestly want you to put an embedded link button on top of this dialogue box.
reply