About

Blog of LLM-Random research group in IDEAS NCBR, started in mid-2022 by Sebastian Jaszczur. Team members, in the order of joining, listed below. We are happy to collaborate with outside institutions and people.

• Sebastian Jaszczur
• Marek Cygan , as advisor
• Jakub Krajewski
• Szymon Antoniak (past member)
• Maciej Pióro
• Tomasz Odrzygóźdź (past member)
• Jan Ludziejewski
• Michał Krutul
• Kamil Ciebiera
• Krystian Król

Public repositories of the group are available at on GitHub.

Research Statement

We are interested in improving the state-of-the-art in Large Language Models, as we believe this to be the most impactful research direction. We want to do this by improving the efficiency of those models during both training and inference. Among the techniques we explore in our work are: Mixture-of-Experts (see Mixture of Tokens), fast/speculative decoding (blogpost coming soon), and improving parameter efficiency (see Neuron Recycling).