Chương trình Nghiên cứu AI Safety Mùa Thu 2025

(Cơ hội này sẽ hoàn toàn trong tiếng Anh, do bản chất của chương trình nghiên cứu cần đọc tài liệu tiếng Anh chuyên ngành. Tuy nhiên, bạn vẫn có thể giao tiếp với điều phối viên HAISN bằng tiếng Việt nếu muốn).

Applications is now open - apply here. (mở trong tab mới)

-

Antoan.AI is partnering with Hanoi AI Safety Network (mở trong tab mới) on the Fall Research Fellowship (FRF) to offer undergraduate/graduate students and early-career/mid-career professionals the opportunity to engage in AI safety research and join a community of people working on AI safety. Previous fellows have published their research projects at ICLR (mở trong tab mới), ICML (mở trong tab mới), and AAAI (mở trong tab mới).

FRF projects span safety evaluations, interpretability, AI control, adversarial robustness, and more. You will have the chance to express your interests and preferred project further into the applications.

The FRF will run for 1-3 months starting August, depending on the nature of your project and participants’ availabilities. The FRF can be participated part-time and remotely. Funding is available for GPU compute and AI subscriptions.

You might be a good fit if:

• You are interested in making advanced AI systems safe and trustworthy.
• You have a technical background (e.g. ML, CS, cybersecurity, maths, physics, neuroscience, etc).
• You are curiosity-driven and willing to pursue your own ideas.

Plus points (although not a requirement)

• You have or plan to pursue a career in research.
• Deep understanding of AI safety arguments and landscape.

The application process includes filling out a short form and a chat with one of our organisers. We will review applications on a rolling basis, and the deadline to apply is Friday August 8th 23:59 AOE. For more information, fill out this form. (mở trong tab mới)

Example AI safety projects below. Feel free to express interest in any of these, or propose your own!

• AI models are really good at knowing where you are from a single image (mở trong tab mới), or inferring your personal information purely from a paragraph of text (mở trong tab mới). This is a concerning dangerous capability, both from a privacy perspective, and from our lack of understanding of it. Can we develop a better understanding of this phenomenon and models’ full capabilities? For example, can they guess locations of indoor images as well? Can we use circuit tracing (mở trong tab mới) to understand how they arrived at the guesses? Do their explanations match this, or are they unfaithful (mở trong tab mới)?
• Current AI systems often know when they are being evaluated (mở trong tab mới). This is concerning, as it undermines the reliability of safety evaluations and allows for strategic deception, such as sandbagging (mở trong tab mới) (intentionally doing badly on evaluations). Blackbox approaches, i.e. simply asking the models when they are aware, might introduce unwanted signals, or might not be effective if models are sandbagging. Prior (mở trong tab mới) research (mở trong tab mới) points to interpretability tools as a promising avenue. Can we create model organisms (mở trong tab mới) of evaluation awareness sandbaggers through finetuning, then demonstrate that whitebox approaches such as probing (mở trong tab mới) or noising (mở trong tab mới) outperform purely blackbox?
• The future will likely include many AI agents running wild on the internet. Currently, it is known that if you put multiple AI agents into the same situation, weird and concerning things happen (infectious (mở trong tab mới) jailbreaks (mở trong tab mới), Claude bliss attractor (mở trong tab mới), etc). Can we develop a more comprehensive understanding of multi-agent dynamics, for example, by using activation engineering to discover cooperative traits (mở trong tab mới), or by finding instances of ‘attractor states’ in misaligned models (mở trong tab mới)?