Preparing Tiled City Model Resources

One of the main functions of the cityschema framework is the Repository/Catalog which provides access to detailed city model resources in various formats compatible with design-oriented 3d modeling tools, open-source tool-chains and game engines and platform-independent preservation of model assets.

Topic Index

Download Latest Version

Pre-Release Notice: This documentation and the associated workspace and tools are almost complete. They require one more run-through before we are sure that they are ready.

Tiled framework for cross-platform collaboration

Exchanging precisely located 3d model data between diverse applications is one of the big challenges of collaborative city model projects. Few non-GIS tools are capable of dealing with models that are both highly detailed and spatially extensive. Most design or history-oriented projects are concerned with neighborhoods rather than entire cities. Segmenting the city model into 5000-foot square tiles creates a framework of snap-together modules that include terrain mesh, drapable groundplan images, and building model collections that are precisely registered in a common, CAD-friendly coordinate system.

There are a variety of exchange formats for 3d models extant in the marketplace. We have chosen the Wavefront OBJ format as the most commonly imported open format that supports a simple triangulated profile and potential to carry image-based texture maps.

The CityTiler Workspace

The CityTiler workspace is an ArcGIS Pro workspace that provides source material, and tools that produce the tile grid and two types of tiled data-sets. Following the convention of the citySchema workflow folder, the cityTiler workspcae is a folder with subfolders for sources and tools and ArcGIS project and layer files among other things.

Tools, Tasks, Maps and Layouts in the CityTiler Workspace

To understand the main components of the CityTiler workspace it wil be useful to open the ArcGIS Pro project, from ArcDocs/CityTiler.aprx and look for the following assets in the Contents and Tasks panes.

  • cityTiler Tools: The heart of the CityTiler workspace is a set of customized geoprocessing models stored in Tools\Bos3d_CityTiler.atbx. These tools are organized into numbered tool groups. Each set of tools is associated with one or more maps, which can be looked at in the ArcGIS contents pane. It is generally better to run these tools through their associated tasks (described below). In the event a tool needs to be repaired or modified, it may be useful to open them form the table of contents. Many tools have two components: one that is designed to produce a data-set for a single tile, and an iterator model that loops through the single-model tool for a set of tiles.
  • Control Maps: Maps with names that end with _CTL are used as the control context for a set of tools. Control maps include layers that are referenced in the tools and therefore should be the active maps when certain tools are being run. Many control maps include a tile selector layer tht is used to specify a specific tile or set of tiles that a tool should operate over.
  • Groundplan Maps: Maps with names that end in _GP or _Map are used to provide symbolization and styling for groundplan images and city-scale reference maps.
  • Tiled Groundplan Layout: In the Layouts section of the contents pane, the Groundplans_LYT provides the scale aand page-size parameters that are used to produce high-resolution tiled groundplan images based on the stylized data provided by the groundplan maps described above. The shift from one tile to another is accomplished by the Tools\python\export_gp_image_tile.py script. Metro TileGrid Layout: The reference map for the tile grid and the fallback context map used by leaflet applications are clipped exactly to the tilegrid boundary. This layour provides the page dimensions and scale that produce the right pixel-size and line weights for these maps.
  • CityTiler Tasks: ArcGIS Tasks are a means of adding essential documentation and controling geoprocessing tools.

The Demo Repository Catalog and Repocat_Data Folder:

The Repository Catalog is a file-system directory that holds all of the tiled data products produced by the cityTiler workflows. For ease of demonstration and demo version of the repoCat folder is provided in the cityTiler workspace. When using these tools in production you may want to keep the working RepoCat folder outside of this workspace.

Navigate to cityTiler/RepoCat/RepioCat_Data and look at the various folders. Most of these are folders that are waiting to hold tiled data that wil be created by the various workflows described below. Noticed that each folder contains a metadata folder with a PDF file inside. This combination of assets and metadata embodies the concept of an Archival Information Package as described on the CitySchema Repository Catalog page

Navigate into the cityTiler/RepoCat folder and open the Bos3d_RepoCat.htm file in your favorite browser. Click the various info-dots related to each of the tiled assets. Notice how these links are fulfilled by the PDF documents saved with each collection of tiled resources.

  • Click on tile I-3 on the clickable tile map. The blue info-dots here link to the same PDF documents. The links to tiled data-sets refer to the zip archives in each of the RepoCat_Data folders.

    Product-Oriented Workflows Executed through from CityTiler Workspace

    The work of producing tiled data-sets begins in the cityTiler workspace. Several datasets are produced, some of which are end products, such as groundplan images. Some of the products are, like the tile grid and the GIS-based terrain resources are used in the tiling process or by the FME workbenches described below.

    Because the FME workbenches are set up to draw terrain mesh and DTM data from existing folders, it is easiest to re-use the folder-names already existing in cityTiler/RepoCat/RepoCat_Data as as the output folders for these various tools.

    • Tile Grid and Associated Points as Shapefiles and DXF and Reference Image: The tilegrid polygons and points along with their special offset projection provide the registration framework that enables easy exchange of precisely geo-located models among tools that do not support formal georeferencing.

      The custom geoprocessing models that create the tile grid polygons and points are provided in the toolbox, 1. Create Tile Grid.

    • Tiled Terrain Resources as Shapefiles and ArcGIS TIN and DXF format: A vector-based digital terrain model (DTM) is an efficient means of recording elevations along breaks-in grade and other critical points. Much more efficient and precise than rasters or contour representations. DTMs provide the most efficient starting place for creating and modifying triangulated terrain meshes. The task, Create Clipped Terrain Datasets produces a folder of clipped DTM data as shape files in a folder that you designate.

      The existence the Tile_Frame and Slim_Tin_Triangles shape files within folders RepoCat_Data/Bos3d_TerrainDTM_SHP/[Tile Label]_GIS is necessary for creating the tiled terrain mesh obj and sketchup format models.

    • Tiled Groundplan Images: Recent true orthophotography or stylized planimetric groundplan maps are useful for painting essential contextual information onto the terrain models to complete the setting for models of structures. The images are georeferenced with ArcGIS world files. Being clipped precisely with the same footprint as the terrain model, these images snap into place onto the terran models in any 3d modeling application.
    • Tiled Vector Groundplan in DXF format: Design studies usually involve reconfiguration of groundplan elements like pavement edges, open spaces and water features. Vector groundplan information in DXF format provides designers with a starting place for their new groundplan ideas -- which also happens to assure that the groundplan patches that are provided with tjheir submitted models wil be coherent with the greater context in the mother model.

    FME Workbenches

    FME provides many capabilities for turning the products of the cityTiler workspace into formats suitible for importing into non-gis platforms and for long-term asset management. Custom FME workbenches have been created to support workflows for generating resources that are indispensible for long term data preservation and interoperability with design-oriented 3d modeling tools.

    The FME workbenches are stored in the CityTiler/tools/FME_Tiling_Tools folder. There is a dated Readme file included with the tools that you can use to compare the issue date of this set of tools against the latest available download listed at the top of this page.

    • Draped_Terrain_OBJ.fmw: Takes a specified tile identiifer and produces a terrain mesh that is draped with a specified groundplan image.

      Accepts the following inputs:

      • Tile_ID The tile to be converted. (Example: BOS_I_3)
      • Folder containing tiled DTM Resources: This folder should contain a folder of tiled terrain resources that includes a folder for each tile. Each of these tiled folders contains a shapefile of the TIle Polygon TIN Triangles.
      • Folder of Tiled Groundplan Images: A folder that contains a sub-folder for each tile. Each of these tile folders contains a Tiled Groundplan Image that will be draped on the terrain mesh for the subject tile.
      • Destination OBJ Folder The name of the folder that will contain the tiled TerrainMech_OBJ datasets.

      Private parameters specify the names of the Tiled Terrain Resources and Groundplan Image folders which are both expected to be in the specified RepoCat_Data folder. In the even that you want to specify other terrain triangles or groundplan images you wil need to edit these private parameters.

      Before running this tool, or the iterator version, you should check RepoCat_Data to make sure that there is an Bos3d_Groundplan_2011_JPG and a Bos3d_TerrainDTM_SHP\[Tile_ID]_GIS dataset for each tile that you intend to produce.

    • Draped_Terrain_OBJ_Iterator.fmw: This workbench gets a list of the eligible tiles by grabbing a list of the folders in the Tiled Terrain Resources folder, it converts the folder names into Tile_IDs and runs the Draped_Terrain_OBJ.fmw for each tile. Before you run this, you should test the Draped_Terrain_OBJ.fmw workbench on a single tile first.

      User Parameters:

      • GIS TerrainDTM Folder for Tile List: A folder containing several subfolders. Each subfolder contains a [Tile_ID]_TerrainDTM_2011_SHP folder that contains a tile boundary and tin triangles shapefiles. This folder is scanned and the sub-folder names are parsed to create a list of tile names which will be passed to the Draped_Terrain_OBJ.fmw workbench.
    • Multipatch_to_ModelStore_Archive.fmw: OBJ-Format Building model collections are created form the geodatabase editions that are generated in the Model Management Workflow.

      This workbench takes the following parameters:

      • RepoCat_Data: folder for output
      • Building Collection Geodatabase: A geodatabase model collection edition produced by the ModelManagement workflow. For your convenience a demo geodatabase has been procided in cityTiler/sources/Bos3d_ModelMgt folder.
      • Tile_ID: if you want to produce a model store collection for a specific tile, enter it here (Example: BOS_I_3). Or leave it blank to convert all of the models within the date range.
      • Begin Date: If you want to just export recent models, enter a date string here as YYYYMMDD. To remove the bottom limit, enter 00000000.
      • End Date: This date should be the date string used in the name of the geodatabase edition you are using.
      • Full or Recent: If you are only selecting recent updates, choose Recent. The default is Full. This value affects the name of the collection that appears in the ModelFinder for each model store.
      This workbench validates and repairs model geometry where necessary and creates a log reporting the errors that it found. It produces a set of tiled model collections for models that are in the Existing, Approved, and History feature classes. Optionally, the Internal and Alt feature classes can be written to a city-wide Internal model store.

      Each tiled or city-wide modelstore includes a catalog database in geojson (polygon) or CSV format.

    • Tiled_SketchUp_Model.fmw: Sketchup format is not an open format, yet it is importable into many design tools and has the advantage of being able to carry an object hierarchy and layers. This saves many steps compared with importing thousands of individual OBJ models into your favorite 3d modeling program. This workbench makes a SketchUp model for a single tile. It uses TIN triangles from the the tiled terrain resources and a geodatabase model collection and produces an integrated Sketchup model that has the buildings grouped according to the value of their Model_ID and categorized into layers based on the value of their status class (Existing, Approved, or History). These models are very convenient as a setting for editing existing building models or creating new development scenarios.

      Accepts the following inputs:

      • Tile_ID The tile to be converted. (Example: BOS_I_3)
      • Folder containing tiled DTM Resources: This folder should contain a folder of tiled terrain resources that includes a folder for each tile. Each of these tiled folders contains a shapefile of the TIle Polygon TIN Triangles.
      • Folder of Tiled Groundplan Images: A folder that contains a sub-folder for each tile. Each of these tile folders contains a Tiled Groundplan Image that will be draped on the terrain mesh for the subject tile.
      • Building Collection Geodatabase: A geodatabase model collection edition produced by the ModelManagement workflow. For your convenience a demo geodatabase has been procided in cityTiler/sources/Bos3d_ModelMgt folder.
      • Issue Date: This is normally the same issue date that identifies the Geodatabase Model Collection that was used.
      • Destination Sketchup folder The name of the folder that will contain the tiled TerrainMech_OBJ datasets.

      Private parameters specify the names of the Tiled Terrain Resources and Groundplan Image folders which are both expected to be in the specified RepoCat_Data folder. In the even that you want to specify other terrain triangles or groundplan images you wil need to edit these private parameters.

      Before running this tool, or the iterator version, you should check RepoCat_Data to make sure that there is an Bos3d_Orthophoto_2023_JPG and a Bos3d_TerrainDTM_SHP\[Tile_ID]_GIS dataset for each tile that you intend to produce.

      This workbench will create a folder named \Bos3d_CityModel_[Issue_Date]_SKP in RepoCat_Data.

    • Tiled_SketchUp_Model_Iterator.fmw: This workbench gets a list of the eligible tiles by grabbing a list of the folders in the Tiled Terrain Resources folder, it converts the folder names into Tile_IDs and runs the Draped_Terrain_OBJ.fmw for each tile. Before you run this, you should test the Draped_Terrain_OBJ.fmw workbench on a single tile first.

    RepoCat Prep and Zip Tools

    The tiled resources produced by the CityTiler workflows and the FME workbenches described above are organized in folders identified by the value of each Tile Identifier. It is a good idea to look at these and to open at least one of each in the appropriate tool to make sure that the pieces are all there and operating as expected. Once you are sure that they are workng, the final step is to zip them. In the case of the model collections, there is a matter of copying the ModelFinder.html and associated resources to each collection. Without the ModelFinder, these collections are nothing but folders filled with randomly-named files.

    The python scripts kept in cityTiler\tools\RepoCat_Prep_Zip_Tools are provided to make it easy to create a zip archive each tiled resource folder. These tools work by dragging and dropping a folder onto the tool in question.

    • Master_ModelFinder: A folder containing the ModelFinder.htm and the associated catalog_resources. These files use the ctalog_jsonp.js file in each model store collection rendering it as a finding aid that allows for identifying and accessing individual models.
    • PrepZip_ModelStore.py: Copies the master ModelFinder.htm and associated resources into a modelstore folder and the prepares a zip archive.
    • ZipChildFolders.py: Creates a zip archive of each of the child folders within a specified parent folder.
    • Rename_Files_Folders: A program for renaming files and folders. Useful in the event that you decide to change the names of resources. This script does not operate via drag and drop interface and requires customization to specify how it will operate.