In this article we propose the first multi-task benchmark for evaluating the performances of machine learning models that work on low level assembly functions.While the use of multi-task benchmark is a standard in the natural language processing (NLP) field, such practice is unknown in the field of assembly language processing.However, in the latest years there has been a strong push in the use of deep neural networks architectures borrowed from NLP to solve problems on assembly code.A kids PCA hats first advantage of having a standard benchmark is the one of making different works comparable without effort of reproducing third part solutions.The second advantage is the one of being able to test the generality of a machine learning model on several tasks.
For these reasons, we propose BinBench, a benchmark for position-aids binary function models.The benchmark includes various binary analysis tasks, as well as a dataset of binary functions on which tasks should be solved.The dataset is publicly available and it has been evaluated using baseline models.