Drift-Aware Byte-Utility Placement for Tiered Embedding Memories in Recommendation Models

Minji Anderson; Aiden Kapoor

doi:10.54097/a09qfn71

Authors

Minji Anderson Department of Computer Science and Engineering, The Ohio State University, Ohio, United States
Aiden Kapoor Department of Computer Science and Engineering, The Ohio State University, Ohio, United States

DOI:

https://doi.org/10.54097/a09qfn71

Keywords:

Recommendation systems, Embedding compression, Nonstationary workload;, Memory hierarchy, Online placement, Migration cost

Abstract

The memory footprint of categorical embeddings is often the dominant cost of neural recommendation. Frequency-aware compression and static placement exploit skew, but a placement chosen from historical frequencies can become inefficient as popular IDs change. We study an online placement extension of a tiered hot/warm/cold embedding layer that evaluates saved slow-memory bytes rather than only cache-hit mass and penalizes row migration. We implement full-dimensional hot rows, projected low-dimensional warm rows, and quotient–remainder cold rows in an executable recommendation model. In a fully specified synthetic dataset of 128,000 chronologically ordered user–item events, three training seeds yield a mean test AUC of 0.6842 at 16× item-table compression, compared with 0.6854 for a full item table. Trace-driven replay at 8× and a fixed cache capacity gives 27.37 slow bytes/request for static access-density placement and 25.73 for the churn-aware controller, including migration traffic. An exact LRU cache remains stronger on this workload (17.33 bytes/request). Sixteen independent synthetic traces per drift regime characterize stability across stationary, abrupt, gradual, and cyclic popularity, with significant variation in benefit. The study isolates a testable byte-utility placement mechanism but does not substitute for public-data or measured-hardware validation.

Downloads

Download data is not yet available.

References

[1] Naumov, M., Mudigere, D., Shi, H.-J. M., Huang, J., Sundaraman, N., Park, J., Wang, X., Gupta, U., Wu, C.-J., Azzolini, A. G., Dzhulgakov, D., Mallevich, A., Cherniavskii, I., Lu, Y., Krishnamoorthi, R., Yu, A., Kondratenko, V., Pereira, S., Chen, X., … & Smelyanskiy, M. (2019). Deep learning recommendation model for personalization and recommendation systems. arXiv, arXiv:1906.00091. https://doi.org/10.48550/arXiv.1906.00091

[2] Yue, X., & Zhu, R. (2024). An explainable hybrid machine learning framework for cash-flow-conditioned multi-horizon liquidity risk early warning: An SME-oriented benchmark study. World Journal of Management Science, 2(1), 63-72.

[3] Acun, B., Murphy, M., Wang, X., Nie, J., Wu, C.-J., & Hazelwood, K. (2021). Understanding training efficiency of deep learning recommendation models at scale. In Proceedings of the IEEE International Symposium on High-Performance Computer Architecture (HPCA) (pp. 802-814). IEEE. https://doi.org/10.1109/HPCA51647.2021.00072

[4] Cheng, H.-T., Koc, L., Harmsen, J., Shaked, T., Chandra, T., Aradhye, H., Anderson, G., Corrado, G., Chai, W., Ispir, M., Anil, R., Haque, Z., Hong, L., Jain, V., Liu, X., & Shah, H. (2016). Wide & Deep learning for recommender systems. In Proceedings of the 1st Workshop on Deep Learning for Recommender Systems (DLRS) (pp. 7-10). ACM. https://doi.org/10.1145/2988450.2988454

[5] Guo, H., Tang, R., Ye, Y., Li, Z., & He, X. (2017). DeepFM: A factorization-machine based neural network for CTR prediction. In Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI) (pp. 1725-1731). https://doi.org/10.24963/IJCAI.2017/239

[6] Rendle, S. (2010). Factorization machines. In Proceedings of the IEEE International Conference on Data Mining (ICDM) (pp. 995-1000). IEEE. https://doi.org/10.1109/ICDM.2010.127

[7] Shi, H.-J. M., Mudigere, D., Naumov, M., & Yang, J. (2020). Compositional embeddings using complementary partitions for memory-efficient recommendation systems. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) (pp. 165-175). ACM. https://doi.org/10.1145/3394486.3403059

[8] Ginart, A. A., Naumov, M., Mudigere, D., Yang, J., & Zou, J. (2021). Mixed dimension embeddings with application to memory-efficient recommendation systems. In Proceedings of the IEEE International Symposium on Information Theory (ISIT). IEEE. https://doi.org/10.1109/ISIT45174.2021.9517710

[9] Sethi, G., Acun, B., Agarwal, N., Kozyrakis, C., Idreos, S., & Ferdman, M. (2022). RecShard: Statistical feature-based memory optimization for industry-scale neural recommendation. In Proceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) (pp. 344-358). ACM. https://doi.org/10.1145/3503222.3507777

[10] Adnan, M., Maboud, Y. E., Mahajan, D., & Nair, P. J. (2021). Accelerating recommendation system training by leveraging popular choices. Proceedings of the VLDB Endowment, 15(1), 127-140. https://doi.org/10.14778/3485450.3485462

[11] Zhou, F. (2026). Moment-constrained side-force allocation for wind-assisted merchant ships: Hydrodynamic performance of retractable anti-leeway fins and flap rudders. Innovation and Technology Studies, 3(1), 27-41.

[12] Zhang, C., Deng, Y., Du, Y., Han, S., Li, Y., & Zhao, Y. (2020). Model size reduction using frequency based double hashing for recommender systems. In Proceedings of the 14th ACM Conference on Recommender Systems (RecSys) (pp. 521-526). ACM. https://doi.org/10.1145/3383313.3412227

[13] Yin, C., Acun, B., Liu, X., & Wu, C.-J. (2021). TT-Rec: Tensor train compression for deep learning recommendation model embeddings. In Proceedings of Machine Learning and Systems (MLSys) (Vol. 3). MLSys.

[14] Kang, W.-C., Cheng, D. Z., Chen, T., Yi, X., Lin, D., Hong, L., & Chi, E. H. (2021). Learning to embed categorical features without embedding tables for recommendation. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) (pp. 840-850). ACM. https://doi.org/10.1145/3447548.3467304

[15] Zhao, X., Liu, H., Fan, W., Liu, H., Tang, J., Wang, C., Chen, M., Zhang, X., Wang, X., & Wen, J.-R. (2021). AutoDim: Field-aware embedding dimension search in recommender systems. In Proceedings of the Web Conference (WWW) (pp. 3015-3022). ACM. https://doi.org/10.1145/3442381.3450124

[16] Liu, S., Gao, C., Chen, Y., Jin, D., & Li, Y. (2021). Learnable embedding sizes for recommender systems. In Proceedings of the International Conference on Learning Representations (ICLR). arXiv, arXiv:2101.07577.

[17] Mudigere, D., Hao, Y., Huang, J., Jia, Z., Tulloch, A., Sridharan, S., Liu, X., Ozdemir, M., Nie, J., Park, J., Ping, L., Xiao, S., Yang, Z., Xing, Y., & Rao, V. (2022). Software-hardware co-design for fast and scalable training of deep learning recommendation models. In Proceedings of the 49th Annual International Symposium on Computer Architecture (ISCA) (pp. 993-1011). ACM. https://doi.org/10.1145/3470496.3533727

[18] Zhao, W., Xie, D., Jia, R., Qian, Y., Song, R., Hu, Y., & Zhang, J. (2020). Distributed hierarchical GPU parameter server for massive scale deep learning ads systems. In Proceedings of Machine Learning and Systems (MLSys) (Vol. 2). MLSys.

[19] Che, H., Tung, Y., & Wang, Z. (2002). Hierarchical web caching systems: Modeling, design and experimental results. IEEE Journal on Selected Areas in Communications, 20(7), 1305-1314. https://doi.org/10.1109/JSAC.2002.801752

[20] Breslau, L., Cao, P., Fan, L., Phillips, G., & Shenker, S. (1999). Web caching and Zipf-like distributions: Evidence and implications. In Proceedings of the IEEE INFOCOM Conference (pp. 126-134). IEEE. https://doi.org/10.1109/INFCOM.1999.749260

[21] Wang, J., Ge, Y., Liu, J., & Jiang, B. (2024). Early warning of liquidity stress in U.S. banks with seasonally aligned frailty hazards and delayed-feedback conformal alarms. Journal of Trends in Finance and Economics, 1(1), 85-100.

[22] Duchi, J., Hazan, E., & Singer, Y. (2011). Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12, 2121-2159.

[23] He, X., & Chua, T.-S. (2017). Neural factorization machines for sparse predictive analytics. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) (pp. 355-364). ACM. https://doi.org/10.1145/3077136.3080777

[24] Teng, D. (2025). TRIAD: Deadline- and reliability-aware scheduling for post-earthquake structural assessment workflows. World Journal of Engineering Research, 3(6), 55-66.

[25] Zhou, G., Zhu, X., Song, C., Fan, Y., Zhu, H., Ma, X., Yan, Y., Jin, J., Li, H., & Gai, K. (2018). Deep interest network for click-through rate prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) (pp. 1059-1068). ACM. https://doi.org/10.1145/3219819.3219823

[26] Shen, A., & Huang, S. (2022). A systems-aware framework for intelligent large-scale recommendation models. Journal of Computer Science and Electrical Engineering, 4(1), 36-44. https://doi.org/10.61784/jcsee4153

[27] Shen, Z., Zhao, W., Wang, B., Wang, Z., & Shang, W. (2026). CAGR: A cross-accelerator graph optimization framework for efficient recommender system inference. IEEE Access, 14, 1-15. https://doi.org/10.1109/ACCESS.2026.3671798

[28] Zhang, F., Shen, A., & Huang, S. (2026). TRIAD: Co-designing mixed-precision quantization, operator fusion, and on-chip memory reuse for real-time vision-language-action inference on automotive NPUs. Results in Engineering, Article 113673. https://doi.org/10.1016/j.rineng.2026.113673