Industry vs Academia: Running a Course on Transformers in Two Setups

The paper addresses the challenge of teaching Transformer architectures to learners with Python, Machine Learning, and Deep Learning experience but little or no exposure to Transformers. The course targets diverse groups from academia and industry, aiming to bridge the gap between theoretical foundations and practical applications. It covers a comprehensive overview of Transformer NLP applications and their extension to other data types. The course structure includes 15 sessions, each with a lecture and a practical part, and two homework assignments organized as CodaLab competitions. The first six sessions focus on the Transformer architecture and its variants (encoders, decoders, encoder-decoders) and model tuning techniques. Subsequent sessions cover multilingualism, multimodality (texts and images), efficiency, event sequences, and tabular data. The course was first run in 2022 and iteratively updated until 2024 with emerging findings on GPT-4, LLMs, RLHF, etc. It has been delivered six times: four times in industry and twice in academia, receiving positive feedback from both audiences.

The course was delivered six times: four times to industry audiences and twice to academic students. Feedback from both groups was positive, indicating the effectiveness of the curriculum and the practical competition format. The iterative updates ensured relevance, incorporating recent developments such as GPT-4, LLMs, and RLHF. The course successfully catered to learners with varying backgrounds, as evidenced by the positive reception. No quantitative metrics (e.g., test scores, completion rates) are reported in the abstract, but the qualitative feedback suggests high satisfaction. The course's dual focus on theory and practice, with competitions providing hands-on experience, likely contributed to its success. The authors note that the course was constantly updated, reflecting the fast-paced nature of the field.