Urban Cultural Inheritance: Generative Adversarial Networks (GANs) Assisted Street Facade Design in Virtual Reality (VR) Environments Based on Hakka Settlements in Hong Kong

Jiahua Dong, Qingrui Jiang, Anqi Wang, Yuankai WangProceedings of the International Conference on Computer-Aided Architectural Design Research in Asia 2023 · DOI

TL;DR

This paper proposes a novel participatory design workflow that uses Generative Adversarial Networks (GANs) and Virtual Reality (VR) to preserve the architectural heritage of Hong Kong’s Hakka settlements. By training a CycleGAN model to translate between modern high-density facades and traditional Hakka styles, the researchers created an immersive digital twin where citizens can interactively "re-skin" urban environments, bridging the gap between rapid modernization and cultural inheritance.

What Problem Does This Paper Address?

Hong Kong is one of the most densely populated cities on Earth. Since the 1950s, rapid urbanization has prioritized high-density housing, leading to the gradual abandonment of indigenous Hakka settlements—villages with over 200 years of history in agriculture and fishing. These sites are increasingly becoming "ghost towns" or are being demolished for new developments.

The challenge for urban designers is twofold: how to preserve the "intangible" cultural identity of these spaces during regeneration, and how to involve the community in that process. Traditional urban planning is often a top-down approach involving only professionals. This paper seeks to democratize the process by providing a tool that allows non-experts to visualize and co-design historical "style infilling" for modern streets.

Methods

The researchers developed a three-step workflow: Data Collection, GAN Modeling, and HCI (Human-Computer Interaction) in VR.

  1. Style Transfer with CycleGAN: The authors chose CycleGAN, a type of GAN designed for "unpaired" image-to-image translation. Unlike standard GANs that require direct A-to-B mapping (e.g., a sketch and its corresponding photo), CycleGAN can learn the stylistic relationship between two distinct sets of images—in this case, modern HK apartment facades and traditional Hakka village facades.
  2. Loss Functions: To ensure the generated facades looked realistic while maintaining the structure of the original building, the model utilized a combination of GAN loss, cycle-consistency loss (ensuring an image translated from A to B and back to A remains the same), and identity loss.
  3. VR Integration: The generated facade options were integrated into a "digital twin" of a Hong Kong street. The technical stack included Blender for 3D modeling, Spoke for scene assembly, and Mozilla Hubs to host the VR environment. This allows users to enter the scene via VR headsets (like the Oculus) or web browsers to select and project different AI-generated facade styles onto buildings in real-time.

Datasets

The study utilized two primary datasets scraped from Google Street View:

  • Set A: Modern Hong Kong high-density residential facades.
  • Set B: Historical Hakka settlement facades.

The researchers curated a training set of 200 images and a validation set of 20 images. To improve model performance, they pre-processed the images using OpenCV for camera calibration (correcting lens distortion) and applied data augmentation techniques like mirroring and rotation to artificially expand the small dataset.

Key Insights and Ideas

  • AI as a "Speculative Designer": Rather than replacing the architect, the GAN acts as a "suggestive tool" that provides design options beyond typical human intuition, blending two disparate architectural languages into a "mixed-style" facade.
  • From Data Subjects to Actors: A major theme is shifting the role of the citizen. Instead of just being "data" for planners to analyze, citizens become "actors" in a co-learning process, using the VR interface to express their preferences for how their cultural heritage should be manifested.
  • The Power of the Digital Twin: By using a digital twin, the researchers can simulate urban regeneration scenarios without the cost or permanence of physical construction, allowing for iterative community feedback.

Key Results

  • Model Performance: The researchers tested the model over 100, 200, and 300 epochs, finding that 300 epochs provided the best "generated power" for creating recognizable Hakka-style features on modern building skeletons.
  • Functional Prototype: The team successfully deployed a multi-user VR platform. Users can take a virtual tour, explore buildings, and use a "Generating Process" interface to select and map AI-generated facades onto 3D models using UV mapping functions.
  • Qualitative Impact: The workflow demonstrates a viable path for "bottom-up" community engagement, potentially increasing citizen awareness and a sense of belonging in contemporary urban environments.

Limitations and Future Work

The authors acknowledge several hurdles:

  • Dataset Scale: With only 200 training images, the model is at risk of overfitting. Future work involves enlarging the dataset and adding labeled images.
  • Interface Complexity: While the VR environment is immersive, the authors note that the UI needs to be more "beginner-friendly" to truly empower non-technical citizens.
  • Semantic Awareness: The current model treats facades as flat images. The researchers plan to integrate image segmentation networks to label specific architectural elements (windows, doors, ornaments), which would give users more granular control over the design process.

Summary updated

Abstract

In Hong Kong, the Hakka settlements are the home of indigenous people who have been involved in agriculture and fishing for over 200 years, which has a special place in Hong Kong’s history. However, these settlements are gradually being abandoned as ghost towns due to rapid urbanisation, where the city is progressively constructing high-density habitats to accommodate the exponentially increased population since the 1950s. This challenges designers to rethink means of preserving urban cultural heritage, while engaging in continuous urban regeneration processes. This study investigates workflows to detect historical building styles in one of the most densely-populated cities in the world - Hong Kong - that further deployed in human-computer interfaces in the virtual reality (VR) environment as a collaborative and suggestive design -107958641080
BibTeX
@inproceedings{dong2023urban,
  author = {Jiahua Dong and Qingrui Jiang and Anqi Wang and Yuankai Wang},
  title = {{Urban Cultural Inheritance: Generative Adversarial Networks (GANs) Assisted Street Facade Design in Virtual Reality (VR) Environments Based on Hakka Settlements in Hong Kong}},
  booktitle = {Proceedings of the International Conference on Computer-Aided Architectural Design Research in Asia},
  year = {2023},
  doi = {10.52842/conf.caadria.2023.1.473},
}
Figure