Wrote an open source command shell wrapper for LLMs to use. Just testing it with with writing a website from scratch. Comparing GPT-4 and Claude 3 Opus. The results: Pretty similar, they can write a PHP page, with components, styles, and active elements like a fun quiz on one of the pages.
A real test would be to invent a framework and see if it can do anything on the new framework. Otherwise it’s just a search engine that amalgamates pieces
Video: https://www.youtube.com/watch?v=4TnaJLq0E-o
Article: https://naisys.org/articles/3-dollar-website-gpt4-vs-claude-opus
GitHub repo: https://github.com/swax/NAISYS