WARP: Workload-Aware Residual Profiling for Graph-Equivalent Recommender Inference

Avery Campbell; Emma Yamamoto; Nina Brown

doi:10.54097/6sehae79

Authors

Avery Campbell Department of Computer Science and Engineering, Jacobs School of Engineering, University of California, San Diego, USA
Emma Yamamoto Google LLC, USA
Nina Brown Manning College of Information and Computer Sciences, University of Massachusetts Amherst, USA

DOI:

https://doi.org/10.54097/6sehae79

Keywords:

Recommendation inference, Computation graphs, Workload adaptation, Cost modeling, Embedding fusion, Reproducibility

Abstract

Hardware-aware graph optimization can accelerate recommender inference, but choosing among equivalent execution plans is vulnerable to inaccurate cost estimates and noisy microbenchmarks. We instantiate the bandwidth-ratio, roofline-plus-dispatch plan-ranking mechanism of the CAGR framework as the analytical prior of an executable selector and extend it with two-parameter residual calibration by batch size and categorical access context. Six equivalent embedding/interactions plans are validated numerically and evaluated using frozen parameters on synthetic CPU request streams. In an initial 16-context study, the calibrated selector reaches 2.55× geometric-mean speedup against an unfused eager reference and reduces the held-out latency-prediction error of the CAGR-style prior from 57.2% to 12.6%. A stronger fixed TorchScript baseline, however, achieves 2.60× in those same observations. We therefore perform a separate six-round, randomized-order remeasurement on fresh requests: WARP obtains 1.94×, whereas fixed TorchScript obtains 1.95×. Their relative speedup is 0.991× (descriptive stratified bootstrap interval [0.977, 1.000]). Thus calibration improves latency estimation but does not establish superiority over the strongest fixed candidate in this bounded library. The study records numerical checks, adaptation cost, raw timings, and the conditions under which policy ranking can become unstable. Results concern synthetic requests and one CPU, not production CTR accuracy or heterogeneous accelerators.

Downloads

Download data is not yet available.

References

[1] Guo, H., Tang, R., Ye, Y., Li, Z., & He, X. (2017). DeepFM: A factorization-machine based neural network for CTR prediction. In Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI) (pp. 1725-1731). https://doi.org/10.24963/ijcai.2017/239

[2] Naumov, M., Mudigere, D., Shi, H.-J. M., Huang, J., Sundaraman, N., Park, J., Wang, X., Gupta, U., Wu, C.-J., Azzolini, A. G., Dzhulgakov, D., Mallevich, A., Cherniavskii, I., Lu, Y., Krishnamoorthi, R., Yu, A., Kondratenko, V., Pereira, S., Chen, X., … & Smelyanskiy, M. (2019). Deep learning recommendation model for personalization and recommendation systems. arXiv, arXiv:1906.00091. https://doi.org/10.48550/arXiv.1906.00091

[3] Cheng, H.-T., Koc, L., Harmsen, J., Shaked, T., Chandra, T., Aradhye, H., Anderson, G., Corrado, G., Chai, W., Ispir, M., Anil, R., Haque, Z., Hong, L., Jain, V., Liu, X., & Shah, H. (2016). Wide & deep learning for recommender systems. In Proceedings of the 1st Workshop on Deep Learning for Recommender Systems (DLRS@RecSys) (pp. 7-10). ACM. https://doi.org/10.1145/2988450.2988454

[4] He, X., Liao, L., Zhang, H., Nie, L., Hu, X., & Chua, T.-S. (2017). Neural collaborative filtering. In Proceedings of the 26th International Conference on World Wide Web (WWW) (pp. 173-182). ACM. https://doi.org/10.1145/3038912.3052569

[5] Lian, J., Zhou, X., Zhang, F., Chen, Z., Xie, X., & Sun, G. (2018). xDeepFM: Combining explicit and implicit feature interactions for recommender systems. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) (pp. 1754-1763). ACM. https://doi.org/10.1145/3219819.3220023

[6] Rendle, S. (2010). Factorization machines. In Proceedings of the IEEE International Conference on Data Mining (ICDM) (pp. 995-1000). IEEE. https://doi.org/10.1109/ICDM.2010.127

[7] Koren, Y., Bell, R., & Volinsky, C. (2009). Matrix factorization techniques for recommender systems. Computer, 42(8), 30-37. https://doi.org/10.1109/MC.2009.263

[8] Zhou, F. (2026). Moment-constrained side-force allocation for wind-assisted merchant ships: Hydrodynamic performance of retractable anti-leeway fins and flap rudders. Innovation and Technology Studies, 3(1), 27-41.

[9] Shen, Z., Zhao, W., Wang, B., Wang, Z., & Shang, W. (2026). CAGR: A cross-accelerator graph optimization framework for efficient recommender system inference. IEEE Access, 14, 38544-38562. https://doi.org/10.1109/ACCESS.2026.3671798

[10] Williams, S., Waterman, A., & Patterson, D. (2009). Roofline: An insightful visual performance model for multicore architectures. Communications of the ACM, 52(4), 65-76. https://doi.org/10.1145/1498765.1498785

[11] Chen, T., Moreau, T., Jiang, Z., Zheng, L., Yan, E., Shen, H., Cowan, M., Wang, L., Hu, Y., Ceze, L., Guestrin, C., & Krishnamurthy, A. (2018). TVM: An automated end-to-end optimizing compiler for deep learning. In Proceedings of the 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI) (pp. 578-594). USENIX Association.

[12] Jia, Z., Padon, O., Thomas, J., Warszawski, T., Zaharia, M., & Aiken, A. (2019). TASO: Optimizing deep learning computation with automatic generation of graph substitutions. In Proceedings of the 27th ACM Symposium on Operating Systems Principles (SOSP) (pp. 47-62). ACM. https://doi.org/10.1145/3341301.3359630

[13] Ma, L., Xie, Z., Yang, Z., Xue, J., Miao, Y., Cui, W., Hu, W., Yang, F., Zhou, L., & Zhou, L. (2020). Rammer: Enabling holistic deep learning compiler optimizations with rTasks. In Proceedings of the 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI) (pp. 881-897). USENIX Association.

[14] Zheng, L., Jia, C., Sun, M., Wu, Z., Yu, C. H., Haj-Ali, A., Wang, Y., Yang, J., Zhuo, D., Sen, K., Gonzalez, J. E., & Stoica, I. (2020). Ansor: Generating high-performance tensor programs for deep learning. In Proceedings of the 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI) (pp. 863-879). USENIX Association.

[15] Niu, W., Guan, J., Wang, Y., Agrawal, G., & Ren, B. (2021). DNNFusion: Accelerating deep neural networks execution with advanced operator fusion. In Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation (PLDI) (pp. 883-898). ACM. https://doi.org/10.1145/3453483.3454083

[16] Tillet, P., Kung, H.-T., & Cox, D. (2019). Triton: An intermediate language and compiler for tiled neural network computations. In Proceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages (MAPL) (pp. 10-19). ACM. https://doi.org/10.1145/3315508.3329973

[17] Chen, J., Han, X., Zhang, X., & Yang, Y. (2023). Exact concept-level explanations for early detection of financial misstatements. Eurasia Journal of Science and Technology, 5(2), 34-48.

[18] Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Köpf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., … & Chintala, S. (2019). PyTorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems, 32, 8024-8035.

[19] Ivchenko, D., Medvedev, D., Klinker, S., & Wu, C.-J. (2022). TorchRec: A PyTorch domain library for recommendation systems. In Proceedings of the 16th ACM Conference on Recommender Systems (RecSys) (pp. 482-483). ACM. https://doi.org/10.1145/3523227.3547387

[20] Mudigere, D., Hao, Y., Huang, J., Jia, Z., Tulloch, A., Sridharan, S., Liu, X., Ozdemir, M., Nie, J., Park, J., Ping, L., Xiao, S., Yang, Z., Xing, Y., & Rao, V. (2022). Software-hardware co-design for fast and scalable training of deep learning recommendation models. In Proceedings of the 49th Annual International Symposium on Computer Architecture (ISCA) (pp. 993-1011). ACM. https://doi.org/10.1145/3470496.3533727

[21] Wang, J., Ge, Y., Liu, J., & Jiang, B. (2024). Early warning of liquidity stress in U.S. banks with seasonally aligned frailty hazards and delayed-feedback conformal alarms. Journal of Trends in Finance and Economics, 1(1), 85-100.

[22] Mirhoseini, A., Pham, H., Le, Q. V., Steiner, B., Larsen, R., Zhou, R., Kumar, N., Norouzi, M., Bengio, S., & Dean, J. (2017). Device placement optimization with reinforcement learning. In Proceedings of the 34th International Conference on Machine Learning (ICML), PMLR 70 (pp. 2430-2439). PMLR.

[23] Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017). Proximal policy optimization algorithms. arXiv, arXiv:1707.06347. https://doi.org/10.48550/arXiv.1707.06347

[24] Rotem, N., Fix, J., Abdulrasool, S., Deng, G., Dzhabarov, R., Hegeman, J., Levenstein, R., Maher, B., Meier, N., & Sela, S. (2018). Glow: Graph lowering compiler techniques for neural networks. arXiv, arXiv:1805.00907. https://doi.org/10.48550/arXiv.1805.00907

[25] Lattner, C., Amini, M., Bondhugula, U., Cohen, A., Davis, A., Pienaar, J., Riddle, R., Shpeisman, T., Vasilache, N., & Zinenko, O. (2020). MLIR: A compiler infrastructure for the end of Moore's law. arXiv, arXiv:2002.11054. https://doi.org/10.48550/arXiv.2002.11054

[26] Wang, J., Chen, X., Zhang, Y., Li, H., & Liu, Z. (2022). Merlin HugeCTR: GPU-accelerated recommender system training and inference. In Proceedings of the 16th ACM Conference on Recommender Systems (RecSys) (pp. 534-537). ACM. https://doi.org/10.1145/3523227.3547405

[27] Harper, F. M., & Konstan, J. A. (2015). The MovieLens datasets: History and context. ACM Transactions on Interactive Intelligent Systems, 5(4), Article 19. https://doi.org/10.1145/2827872

[28] Shang, W., Teng, D., Chen, T., Zhang, F., & Ding, J. (2026). Neuro-Elastic: A unified framework for hardware-aware adaptive quantization and dynamic sparsity in real-time edge intent prediction. IEEE Access, 14, 111168-111187. https://doi.org/10.1109/ACCESS.2026.3715276

[29] Zhang, F., Shen, A., & Huang, S. (2026). TRIAD: Co-designing mixed-precision quantization, operator fusion, and on-chip memory reuse for real-time vision–language–action inference on automotive NPUs. Results in Engineering, Article 113673. https://doi.org/10.1016/j.rineng.2026.113673