Resultado de búsqueda
This research workshop gathers academic, industrial and independent researchers from many affiliations and whose research interests span many fields of research across AI, NLP, social sciences, legal, ethics and public policy.
- Blog
The BigScience OpenRAIL-M License 🌸Introducing The World’s...
- ACL 2022
The ACL 2022 workshop "Challenges & Perspectives in Creating...
- bigscience (BigScience Workshop)
BigScience is an open and collaborative workshop around the...
- Blog
BigScience is an open and collaborative workshop around the study and creation of very large language models gathering more than 1000 researchers around the worlds. You can find more information on the main website at https://bigscience.huggingface.co.
BigScience Workshop. 🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading. Toolkit for creating, sharing and using natural language prompts. Central place for the engineering/scaling WG: documentation, SLURM scripts and logs, compute environment and data.
- Overview
- Trainings
Research workshop on large language models - The Summer of Language Models 21
At the moment we have 2 code repos:
1.https://github.com/bigscience-workshop/Megatron-DeepSpeed - this is our flagship code base
2.https://github.com/bigscience-workshop/bigscience - (this repo) for everything else - docs, experiments, etc.
Currently, the most active segments of this repo are:
•JZ - Lots of information about our work environment which helps evaluate, plan and get things done
Train 1 - 13B - unmodified Megatron gpt2 - baseline
•the full spec and discussions •the training script •checkpoints and logs: •tensorboard •logs •chronicles You can watch the training logs live by running this tail -f like script over remote log file that gets synced to the hub once an hour:
Train 3
Architecture and scaling baseline runs: no fancy tricks, just GPT2. Here are links to the respective tensorboards:
Train 8
104B - unmodified Megatron gpt2 - with extra-wide hidden size to learn how to deal with training instabilities •the full spec and discussions •the training script •checkpoints and logs: •tensorboard •logs •chronicles You can watch the training logs live by running this tail -f like script over remote log file that gets synced to the hub once an hour:
🌸Introducing The World’s Largest Open Multilingual Language Model: BLOOM🌸. Large language models (LLMs) have made a significant impact on AI research. These powerful, general models can take on a wide variety of new language tasks from a user’s instructions.
A one-year long research workshop on large language models: the Summer of Language Models 21 🌸
The BigScience workshop is excited to announce that the training of the BigScience language model has officially started. After one year of experiments, discussions, and development to lead up to this, with more than 1000 collaborators worldwide, the model will have 176B parameters trained on data from 46 languages.
