For the weakly supervised anomaly detection task, most existing work is limited to the problem of inadequate video representation due to the inability to model long-time contextual information. We propose a weakly supervised adaptive graph convolutional network (WAGCN) to model the contextual relationships among video segments. And we fully consider the influence of other video segments on the current segment when generating the anomaly probability score for each segment. Firstly, we combine the temporal consistency as well as feature similarity of video segments for composition, which makes full use of the association information among spatial-temporal features of anomalous events in videos. Secondly, we propose a graph learning layer in order to break the limitation of setting topology manually, which adaptively extracts sparse graph adjacency matrix based on data. Extensive experiments on two public datasets (i.e., UCF-Crime dataset and ShanghaiTech dataset) demonstrate the effectiveness of our approach.