Llama3 on Groq

by matanyall on 4/19/2024, 7:04:03 PM with 7 comments

  • by Oras on 4/19/2024, 7:10:35 PM

    That's impressive. I asked to summarise an article in 5 bullet points, and the output was 812.81 T/s on Llama 3 8B.

  • by frozenport on 4/20/2024, 1:48:35 AM

    LLama3 looks particularly good at tool calling

    Groq's low latency is particularly good for tool calling

    Seems like two techs that will make coding obsolete :-)

  • by Alifatisk on 4/20/2024, 10:13:10 AM

    Is the python lib open-source? I could only find the ja lib for Groq.

  • by WhatsName on 4/19/2024, 8:55:16 PM

    What is tbe cost per Mio. Token for llama3 70b on groq?

  • by jacooper on 4/19/2024, 9:06:45 PM

    When is Mixtral 8x22b coming?