In recent years, the Whole Slide Image (WSI) classification task has achieved great advancement due to the success of Multiple Instance Learning (MIL). However, MIL-based methods face two limitations: 1) often select the top-ranking instances of a WSI based on different metrics (e.g., attention score) to train the model due to the large resolution of WSIs, which may lead to missing global information; 2) usually consider all instances within a bag to be unordered, which will cause the local context information to be missing. To address the limitations of MIL-based methods, we formulate the WSI classification task as a long sequence classification problem in a weakly supervised setting. We propose a Noise Robust Memory-augmented (Norma) framework that serializes the WSI into an ordered sequence and caches each segment for future reuse in a sequential manner. By applying such paradiam, global and local context information of a WSI can be obtained during training. Furthermore, Normal adopts a Cyclic Training process to eliminate the noise introduced by the WSI-level labe. We obtains state-of-the-art results on CAMELYON-16, TCGA-BRAC and TCGA-LUNG datasets. We will release the code upon acceptance.