JIAYIN ZHU

zhujiayin@u.nus.edu | Github

EDUCATION

National University of Singapore (NUS)Jan 2023 - Present
Ph.D. Student in Computer Science

National University of Singapore (NUS)Aug 2021 - Jan 2023
Master of Computing (General Track)

• GPA: 4.77/5.0

Beijing Institute of Technology (BIT)Sep 2017 - Jun 2021
Bachelor of Management in Information Management and Information System

• GPA: 90.50/100.00; Ranked: 4/35 & Top 1 /35 in Junior Year
• Dean’s Academic Scholarship (2020); First-class Scholarship (2020); Second-class Scholarship (2017,2018,2019)

University of California, IrvineJun 2020 - Sep 2020
Summer Research

• Topic: Research on Using ML & Semantic Segmentation to Examine Green Space Types Based on Street View Images (Prof. XIE Xiaohui, Dept. of Computer Science)

PUBLICATIONS

3D Magic Mirror: Clothing Reconstruction from a Single Image via a Causal Perspective. [link]

Zhedong Zheng, Jiayin Zhu , Wei Ji, Yi Yang, Tat-Seng Chua.

ACM Multimedia (ACM MM) 2022 (Submitted).

Abstract (+)

This research aims to study a self-supervised 3D clothing reconstruction method, which recovers the geometry shape, and texture of human clothing from a single 2D image. Compared with existing methods, we observe that three primary challenges remain: (1) the conventional template-based methods are limited to modeling non-rigid clothing objects, e.g., handbags and dresses, which are common in fashion images; (2) 3D ground-truth meshes of clothing are usually inaccessible due to annotation difficulties and time costs. (3) It remains challenging to simultaneously optimize four reconstruction factors, i.e., camera viewpoint, shape, texture, and illumination. The inherent ambiguity compromises the model training, such as the dilemma between a large shape with a remote camera or a small shape with a close camera.
In an attempt to address the above limitations, we propose a causality-aware self-supervised learning method to adaptively reconstruct 3D non-rigid objects from 2D images without 3D annotations. In particular, to solve the inherent ambiguity among four implicit variables, i.e., camera position, shape, texture, and illumination, we study existing works and introduce an explainable structural causal map (SCM) to build our model. The proposed model structure follows the spirit of the causal map, which explicitly considers the prior template in the camera estimation and shape prediction. When optimization, the causality intervention tool, i.e., two expectation-maximization loops, is deeply embedded in our algorithm to (1) disentangle four encoders and (2) help the prior template update. Extensive experiments on two 2D fashion benchmarks, e.g., ATR, and Market-HQ, show that the proposed method could yield high-fidelity 3D reconstruction. Furthermore, we also verify the scalability of the proposed method on a fine-grained bird dataset, i.e., CUB.

Using Machine Learning to Examine Street Green Space Types at a High Spatial Resolution: Application in Los Angeles County on Socioeconomic Disparities in Exposure. [link]

Yi Sun, Xingzhi Wang Jiayin Zhu (co-second author), Liangjian Chen, Yuhang Jia, Jean M. Lawrence, Luo-hua Jiang, Xiaohui Xie, Jun Wu.

Science of The Total Environment, 2021.

Abstract (+)

Background: Compared to commonly-used green space indicators from downward-facing satellite imagery, street view-based green space may capture different types of green space and represent how environments are perceived and experienced by people on the ground, which is important to elucidate the underlying mechanisms linking green space and health. Objectives: This study aimed to evaluate machine learning models that can classify the type of vegetation (i.e., tree, low-lying vegetation, grass) from street view images; and to investigate the associations between street green space and socioeconomic (SES) factors, in Los Angeles County, California. Methods: SES variables were obtained from the CalEnviroScreen3.0 dataset. Microsoft Bing Maps images in conjunction with deep learning were used to measure total and types of street view green space, which were compared to normalized difference vegetation index (NDVI) as commonly-used satellite-based green space measure. Generalized linear mixed model was used to examine associations between green space and census tract SES, adjusting for population density and rural/urban status. Results: The accuracy of the deep learning model was high with 92.5% mean intersection over union. NDVI were moderately correlated with total street view-based green space and tree, and weakly correlated with low-lying vegetation and grass. Total and three types of green space showed significant negative associations with neighborhood SES. The percentage of total green space decreased by 2.62 [95% confidence interval (CI): −3.02, −2.21, p < 0.001] with each interquartile range increase in CalEnviroScreen3.0 score. Disadvantaged communities contained approximately 5% less average street green space than other communities. Conclusion: Street view imagery coupled with deep learning approach can accurately and efficiently measure eye-level street green space and distinguish vegetation types. In Los Angeles County, disadvantaged communities had substantively less street green space. Governments and urban planners need to consider the type and visibility of street green space from pedestrian's perspective.

Comprehensive Evaluation of the College Students’ Back-to-school Safety Degree Under the Background of the COVID-19 Epidemic.

Xiong Dehui Zhu Jiayin (co-first author), Li Jiami, Cui Lixin.

IEEE Access (Under Review)

Abstract (+)

Since the outbreak of COVID-19, it has been affecting the normal life of people all over the world. Due to the epidemic, universities adopt the online teaching method. However, this type of teaching mode has a certain adverse effect. It is urgent for university students to return to school. This paper puts forward a comprehensive evaluation of the college students' back-to-school safety degree based on analytic hierarchy process (AHP), which aims to get the students' back-to-school safety degree according to the specific situation of different colleges and universities. Taking the actual data of Beijing Institute of Technology, Tsinghua University, Wuhan University, Shanghai University of Finance and Economics and Chongqing University in June as examples, the results show that the comprehensive evaluation values of the five universities are 0.331 (Level 4), 0.308 (Level 4), 0.741 (Level 2), 0.590 (level 3) and 0.664 (Level 2). The evaluation results of this index system are in line with the reality, and can make a reference contribution to the safety evaluation of colleges and universities during the epidemic period.

Peer Influence in Q&A platforms- Empirical Evidence from Zhihu.

Lin Jia, with Siyuan Li, Sijia Zhang, Jiayin Zhu, Jiami Li.

Informs 2020

Research on Evaluation Indicator System of Customer-perceived Service Quality on WeChat Social Network. [paper]

Jiawei Liu, Yinghui Gao, Lixin Cui, Jiayin Zhu, Yixuan Wu.

Journal of Information and Management, 2019(Z2)

Abstract (+)

Social network can be said to be the third wave of Internet development after network and search engines. Countless examples can be found to prove the influence of social networks on our life. With the advancement of social networking technologies，we have moved our life，work，and entertainment into a new world of social networks，where each of us is a digital immigrant. This paper focuses on the research of WeChat，a representative of the popular mobile social network industry in recent years. Firstly，the paper constructs the initial measurement system for SNS quality by crawling and analyzing 6300 pieces of data from 3 main websites of comments on Weichat. Secondly，combined with a questionnaire survey，the WeChat social network customer perceived service quality evaluation index system is established. . Finally，some strategies and suggestions for improving the service quality on WeChat are put forward.

RESEARCH EXPERIENCE

• Research Intern, Sea-NExT Joint Lab, NUSDr. Zhedong Zheng and Prof. Tat-Seng Chua 3D Image and Video Reconstruction of Clothing via a Causal PerspectiveOct 2021 - Dec 2022

  • Researched the self-supervised 3D reconstruction of human clothing from a single image via a causal perspective; proposed a deep learning model to adaptively reconstruct 3D non-rigid objects from 2D images without 3D annotations
  • Studied existing works, introduced an explainable structural causal map (SCM), and built the model following its spirit; constructed 4 encoders, i.e., shape, camera, texture, and illumination, and explicitly considered the prior template to solve the inherent ambiguity among the encoders
  • Utilized the causality intervention tool, i.e., two expectation-maximization loops, to disentangle the 4 encoders and help the prior template update
  • Conducted experiments on 3 benchmarks, including ATR, Market-HQ, and CUB, and achieved competitive 3D reconstruction results

• Researcher, Prof. Shen's Research Group, BITProf. SHEN Meng Neural Network-Based Modeling for Residential Energy Conservation Intervention During COVID-19 PandemicFeb 2021 - Jun 2021

  • Researched neural network-based modeling for residential energy conservation intervention during the COVID-19 pandemic, studied the impact of different types of interventions on energy-saving behaviors
  • Constructed an LSTM neural network model based on results of a Singapore residential energy-saving intervention experiment, combining features of 4 intervention types, i.e., Covid-19 pandemic, family demographic characteristics, and weather
  • Conducted experiments via counterfactual causal intervention, analyzed effects of intervention and Covid- 19 on family electricity consumption, predicted increase in household electricity consumption caused by Covid-19
  • Extracted the impact of different intervention strategies on household electricity consumption and provided support for government and power grids to formulate energy-saving intervention strategies

• Research Student, UCInspire Program, Dept. Of Computer Science, UC, IrvineProf. Xiaohui Xie Using Machine Learning to Examine Street Green Space and Their Associations with Socioeconomic and Environmental Factors in Los Angeles CountyJun 2020 - Sep 2020

  • Applied a novel deep learning semantic segmentation model-"HRNet + OCR", for classification of 3 types of green spaces based on high-resolution street view images, to examine green space might help understand association of green space and public health
  • Developed high-resolution networks (HRNets) + Object-Contextual Representations (OCR) model for three types of green spaces classification, i.e. tree, low-lying vegetation, and grass
  • Improved model performance by 4% via adopting Focal Loss and Adam optimizer, inferred on 3 million real street view images, achieved a high accuracy with 92.5% mean IoU (Intersection over Union) with 10- fold cross-validation on dataset of 1500 annotated images
  • Reviewed literature on top-ranked Semantic Segmentation models, thoroughly studied model architectures, features, and advantages; summarized comparison of different models

• Research Intern, Prof. Li’s Research Group, School of Computer Science, BITProf. LI Ronghua Establishment of Knowledge Graph in Computer NetworkMay 2020 - Sep 2020

  • Created a knowledge graph in computer network field from scratch, built a website with Neo4j graph database, achieved entity extraction from text, word cloud generation, entity and relationship query
  • Wrote a web crawler using Python Scrapy to gather data relevant to computer network from Baidu Baike and (Hudong) Baike
  • Applied k-nearest neighbors (KNN) to classify all the entities to achieve entity recognition, and built word vectors with fastText to generate word cloud
  • Used Piecewise Convolutional Neural Networks (PCNN) for relation extraction based on the relationship triples crawled from Wikipedia corpus after alignment

• Researcher, Dr. Jia’s Lab, BITAdvisor: Dr. JIA Lin Peer Influence in Q&A platforms- Empirical Evidence from ZhihuJan 2019 - Jul 2020

  • Studied the varying influences of different categories of followees over focal user behaviors, and further investigated the difference of the influence between professional topics and non-professional ones based on data of focal users from Zhihu, China’s Q&A social platform
  • Built a Scrapy-Redis distributed crawler in Python to collect data of focal users mainly interested in professional topic: Stomatology and Internal Medicine, and non-professional topic: Health Preservation and Living Habits, obtained over 4600 samples relating to over 2 billion users
  • Proposed a new taxonomy of followees regarding Q&A patterns, defined the model for answer counts and the ratio of answer to question
  • Analyzed and compared the statistical results of the influence in professional and non-professional topics, verified the rationality of the proposed user category, and concluded that more contents from the followees of focal users would facilitate contribution behaviors
  • Contributed to the poster published in Informs Conference

• Researcher, Prof. Cui’s Lab, BITAdvisor: Prof. CUI Lixin Comprehensive Evaluation of the Safety of College Reopening During COVID-19May 2020 - Jul 2020

  • Employed Analytic Hierarchy Process(AHP) to build a safety index for returning college students during the pandemic; modeled a hierarchy by considering the local COVID-19 information, medical resources for safety, population and epidemic response level, and school-specific information and detailed schedules
  • Analyzed through a series of pairwise comparisons, measured consistency for evaluation, determined the weight of criteria
  • Pre-processed relevant data, calculated safety index of multiple colleges, suggested school-reopening plans
  • Finished a paper as co-first author submitted to IEEE Access

  The Influence of Transactive Memory System on Customer Involved Service Innovation -A Case Study of MIUI.ComJun 2019 - Jun 2020

  • Studied the dynamic relationship between transactive memory system (TMS) and customer-involved service innovation by analyzing the real data from MIUI BBS
  • Built a web crawler with Python to collect and cleanse data from the website MIUI BBS in support of quantitative analysis with Pajek
  • Provided assistance in empirical research using SPSS Amos, including performing reliability and validity analysis, applying Structural Equation Modeling(SEM), Bootstrap, and hierarchical regression analysis for validation and verification of the proposed model

  Evaluation of Customer-perceived Service Quality on WeChatJun 2019 - Feb 2020

  • Studied WeChat social network for customer-perceived service quality evaluation
  • Collected data of 6300 comments on WeChat via a self-built web crawler in Python
  • Assisted in building customer perceived service quality evaluation index system for WeChat, and proposed suggestions for improving service quality based on analysis results

• Research Intern, LI Zhenhua’s Lab, Tsinghua UniversityAdvisor: Prof. LI Zhenhua Study on Solution for Maintaining the Whole Filesystem in Object StorageJul 2019 - Jun 2020

  • Studied the solution for maintaining the whole filesystem in the object storage
  • Investigated existing object cloud storage products and methods; assisted in completing a survey on Quality of Service (QoS) gap and fixes, induced the relaxed consistency gap, and concluded mainstream techniques for maintaining strong consistency

COMPETITION

• Core Member, Finalist(1%) in 2020 Interdisciplinary Contest in Modeling (ICM), COMAP

  Topic: A Model for Projected Plastic Waste Using Multiple Regression AnalysisFeb 2020

PROJECT

• Arduino Rubiks Cube Solving Robot Based on Machine Learning and OpenCV [Github]Oct 2019 - Dec 2019

  • Developed a Rubik’s cube robot which automatically solved the Rubik’s cube using image processing and recognition methods and turned it via stepper motor control using Arduino
  • Applied the UNet model for Rubik’s cube faces’ image segmentation, and used OpenCV to analyze color of the blocks
  • Adopted Two-Phase Algorithm to solve the cube; built a GUI with Tkinter
  • Designed the control circuit with Arduino to drive six 4-phase stepper motors to control the center block

• Design of E-learning SystemMay 2020 - Jul 2020

  • Designed an online education system, EELS (Excellent E-Learning System), which facilitates students taking online courses, completing assignments and exams, communicating with classmates and contacting tutors
  • Finished requirement analysis, drew UML and ERD, designed subsystems for course material uploading, live classes, homework assignments, discussions, examinations, and learning management for instructors

• Wechat Mini-program for Eco-point System in Home Appliance recyclingMar 2019 - Mar 2020 College Students’ Innovation and Entrepreneurship Training Program

  • Designed a Wechat mini-program which offers users with eco-points after they purchase energy-efficiency appliances, offered functions of points redemmation for relevant products; the mini program consisted of 4 functional modules: eco-point accumulation, friends interaction, community and point mall, released to public

SKILLS/OTHERS

• Proficient in Python, R, C, Java, Matlab, HTML/CSS/JavaScript, SQL; SPSS, Stata; Pytorch, Scrapy
• Self-learned: Machine Learning by Andrew Ng (Stanford online Course on Coursera)
• 3rd Prize(Team Leader), Global Management Challenge(GMC), national-level(11/2017, 11/2018)

 

Visit my blog to know more about me!