Purpose: 1) Develop a deep learning algorithm for brain aneurysm detection exploiting weak labels and prior anatomical knowledge. 2) Describe and release the largest Time-Of-Flight Magnetic Resonance Angiography (TOF-MRA) dataset to the community. Materials and Methods: In this retrospective study we retrieved TOF-MRA images of 284 subjects (170 females) scanned between 2010 and 2015. Out of these, 157 are patients with a total of 198 aneurysms, while 127 are controls. We used spherical weak labels as detection ground truth, thus making data annotation, a major bottleneck for medical AI, noticeably faster. Since aneurysms mainly occur in specific locations, we built our deep neural network leveraging the anatomy of the brain vasculature. To assess model robustness, we participated in the first public challenge for TOF-MRA data (93 patients, 20 controls, 125 aneurysms). We stratified results according to aneurysm risk-of-rupture, location, and size. Results: Our network achieves a sensitivity of 80% on the in-house data, with False Positive (FP) rate of 1.2 per patient. On the public challenge data, sensitivity was 68% (FP rate = 2.5), ranking 4th/16 on the open leaderboard. We found no significant difference in sensitivity between risk groups (p = 0.75), locations (p = 0.72), or sizes (p = 0.15). Conclusion: Competitive results can be obtained using fast weak labels and anatomical knowledge for automated aneurysm detection. Our open-source code and open access dataset can foster reproducibility, and bring us closer to clinical application.