reply on: dawn of the token based economy \ stacker news ~econ

pull down to refresh

100 sats \ 3 replies \ @optimism 23 May \ on: dawn of the token based economy econ

I guess small good enough model with lots of unified memory for context could get decently far

I've been a/b testing large models on venice (because they let me pay one-off with sats) vs small models on a macbook for agents. Large models are better at task breakdown and code generation, but even qwen3:4b locally can do pretty amazing things if you instruct it correctly.

You have to break down further for smaller models so you'll be slower, but if you have a 10x better idea that big tech can't steal from you while you're working it, 10x slower don't matter that much: compute sovereignty feels important.

0 sats \ 2 replies \ @optimism 23 May

To clarify "instruct it correctly". During task graph decomposition, this is an example what a small model did locally (took forever) because of a modeling inefficiency in the json output I instructed it to give me:

Now, dependencies: T2, T3, T4, T5, T6 depend on T1. T7 depends on all previous tasks. Parallelizable tasks could be T2, T3, T4, T5, T6 as they are all dependent on T1 but can be done in parallel once T1 is done. However, since they are all after T1, maybe they can't be parallelized until T1 is done. Wait, no. If T1 is done, then T2, T3, T4, T5, T6 can be done in parallel. So parallelizable_tasks would include those. The critical path is T1 followed by T7, but since T2-T6 can be done in parallel, the critical path might be T1, then T7. But the longest path is T1 -> T2 -> T7, T1 -> T3 -> T7, etc. So the critical path is T1, T2, T3, T4, T5, T6, T7. But if T2-T6 can be done in parallel, the total time would be T1 + max(T2, T3, T4, T5, T6) + T7. So the critical path is T1, then T7, but the dependencies are such that T7 depends on all previous tasks. Wait, no. T7 depends on all tasks, but since T2-T6 are parallel, the critical path is T1, then T7. But the actual dependencies are that T7 depends on all tasks, but since T2-T6 are done in parallel, the critical path is T1, then T7. However, the example in the problem shows that the critical path is the sequence of dependent tasks. So in this case, the critical path would be T1, T2, T3, T4, T5, T6, T7, but since T2-T6 can be done in parallel, the critical path is T1, then T7. But the example in the problem shows that the critical path is the sequence of tasks in order. So maybe in this case, the critical path is T1, T2, T3, T4, T5, T6, T7. But with T2-T6 in parallel, the total time would be T1 + (time for T2-T6) + T7. So the critical path is T1, then T7, but the dependencies are that T7 depends on all tasks. Hmm, maybe the critical path is the longest sequence of dependent tasks. So if T7 depends on all previous tasks, then the critical path is T1, T2, T3, T4, T5, T6, T7. But since T2-T6 can be done in parallel, the actual time would be T1 + max(T2, T3, T4, T5, T6) + T7. Therefore, the critical path is T1, then T7, but the dependencies are that T7 depends on all previous tasks. So in the task graph, the critical path would be the sequence from T1 to T7, but with T2-T6 in parallel. The example in the problem's sample output shows that the critical path is the sequence of tasks in order. So perhaps in this case, the critical path is T1, T2, T3, T4, T5, T6, T7. But since T2-T6 can be done in parallel, the actual time is T1 + max(T2, T3, T4, T5, T6) + T7. So the critical path is T1, then T7, but the dependencies are that T7 depends on all previous tasks. Therefore, the critical path is the sequence of tasks in order. So the critical path would be T1, T2, T3, T4, T5, T6, T7. But with T2-T6 in parallel, the actual time is T1 + max(T2, T3, T4, T5, T6) + T7.

Now, the parallelizable tasks are T2, T3, T4, T5, T6, as they can be done in parallel once T1 is done. So the parallelizable_tasks would include those. The critical path is the sequence from T1 to T7, but since T2-T6 are in parallel, the critical path is T1, then T7. But according to the example, the critical path is the sequence of tasks in order. So in the example, the critical path is the sequence of tasks in order. So in this case, the critical path is T1, T2, T3, T4, T5, T6, T7. However, since T2-T6 can be done in parallel, the actual time is T1 + max(T2, T3, T4, T5, T6) + T7. Therefore, the critical path is T1, T7, but the dependencies are that T7 depends on all previous tasks. So the critical path is the sequence of tasks in order. Therefore, the critical path is the sequence from T1 to T7. But the example in the problem shows that the critical path is the sequence of tasks in order, so that's the case here.

100 sats \ 1 reply \ @aljaz OP 23 May

have you looked into combining large action models with llms for orchestration?

there was a research paper explaining the methodology of using large context frontier models as planning and smaller locally hosted models for execution, also used for working with sensitive data since no data was shared to the cloud based model, only processed by local ones. but i can't find it now...

0 sats \ 0 replies \ @optimism 23 May

I've been looking into something like that: I envision a work queue where:

I spin up AWS Inf2 instances on demand that I pack with a large 100B+ instruct tuned reasoning LLM or maybe a LAM (but i'd have to learn more about that first). These do decomposition, review and maybe even prompt tuning?
Local m4 box(es) then run smaller models like devstral or codellama for actual operations.