EnviroScreen Shiny App overview

Landing page for the multiple repositories associated with the Colorado EnviroScreen project.

EnviroScreen Shiny App overview

This document is meant to provide some description context to support active development and troubleshooting of the code base.

Error and Troubleshooting

Troubleshooting a shinyapp can be a bit challenging at time. While developing this application the most common source of error came from miss matched variable names between the input dataset and parameters within the application. I speak about this quite directly throughout the document. Aside from the naming issues the three primary reactive elements; map, histogram, tables do interface with each other but not at the start of the application. This means that specific errors within those features will likely appear on deployment. Usually the error message will point you to the corrent element. If the app runs but fails once a specific request is provided, that’s where to look.

If things are really bad, like they were when we tried the translation for the first time follow the steps below.

  1. Comment out all the content in the server function and make sure the UI looks like you expect it should.
  2. Incrementally introduce a single server side functionally at a time making sure it works as expected before moving on.
    • this is a great way to understand the dependencies and input to the various reactive actions in the application.
  3. once things work locally, test deployment to server.

Deployment

The main branch in the github repository reflects the currently deployed version of the application.

Any addational braches are there for specific development reasons and will be resolved into the main or deleted when appropriate.

Before deploying to the public URL

  1. test application locally.
    • take your time here. Run through tables, geographies, update the map, test map layers.
    • errors with indicator names will only show when a specific indicator is selected.
  2. Push version to test site on end point server.
    • This ensure your application will run on the shinyapps.io server
  3. If it worked on the test site, push to main server.
    • You be only slightly worried about crashing things at thing point because of the previous two steps.

Inputs

  • envoData : primary data generated by the data processing code base. Column names from this dataset are what defines the display of all information within the application. Change a column name without adjusting features in the application and something, probably multiple reactive elements will break.

  • oil : Map visualization feature generated in the data processing code base.
  • coal : Map visualization feature generated in the data processing code base.
  • rural : Map visualization feature generated in the data processing code base.
  • descriptors : CSV containing indicator names, short descriptions, and all content displayed on the indicator descriptions table. Indicator names in this file must match those in the envoData dataset.
  • justice40 : Map visualization feature generated in the data processing code base.
  • di : Map visualization feature generated in the data processing code base.
  • sm : Map visualization feature generated in the data processing code base.

Reactive Objects

These features are what the users actively engage with to make changes to the application. Some are very obvious, things like buttons, others are more subtle, map clicks. Most of these features are defined in the UI but some are actions defined within the widget (map and tables).

Defined in the UI

  • updateMap : Button that allows changes to geographic scale, indicator, and measure/percentile to occur. This button is here to prevent a lag of request. Specifically with the census block level data, it takes a few seconds to update the map visualization.
  • Geom : Used to define the geographic scale displayed on the map, tables, and histograms. This object is used a lot for filtering dataset. Tables and histograms look to this object for changes.
  • Indicator : Alters the layer displayed on the map.
  • percentile : Alters the layer displayed on the map.
  • removeHighlight : Clears the user defined map element that can be added based on table selections.
  • mymap : The map obect. This is the most engaged object within the application. There are two functions that influence it and well as proxy elements. It also ingest all input dataset in some way. The application was built around the map so it tends to be a bit more stable then the histograms/tables. That said because it engages with all the data and most the reactive element it is the best place to start the troubleshooting method.
  • indicatorDesc : Block quote of short indicator descriptions
  • histEnviroScreen : Histogram of the Enviroscreen Score. This features is larger then other histograms. All histograms are defined with the same function, so issues should be evaluated from the function rather then the app.r file.
  • histExposure : Historgram of environmental exposure
  • histEffect : Histogram of environmental effects
  • histClimate : Histogram of Climate
  • histSocial : Histogram of Social Economic
  • histDemo : Histogram of Demographics
  • tableAll : Contructs the data table from which specific tables are displayed.
  • tableSelect : Filters the table data to visualize the information that the user has selected. Note that all but the Indicator Descriptions table are parsed from the same input dataset. Having two different input dataset under this call has created some real challenges in maintaining similar performance between the two features. Currently the table work well for the enviroscreen data, and not well for the reference material in the indicator description table. This is a compromise.
  • button_table : a poorly named variable that creates a map object based on features selected in the table. Currently selections are held across all tables. Selecting a features on the map also highlights the object on a table. The result of this is that there are
  • downloadData : Download the data held on all be the Indicator Description table. Data is limited to the currently selected geography.
  • downloadData2 : Download the data held in the Indicator Description table. This does not change with geography.

inputs from base functionality

  • mymap_shape_click : Leaflet object that collects information about the user selected map feature. This only applies to Indicator Layer. Selecting a DI community area does not effect the reactively of the map elements. THe select does two things automatically. One; histograms are redrawn to show the bin in which the selected feature was identified. Two; the table data is sorted so that the select object is at the top of the table. The table is sorted by the GEOID at that point. Same as aphabetically, except that some census tracts/block groups can appear above the selected GEOID. This measure that selecting a census tract from adams county does not garuntee that ll census blocks from adams county will be visable near the start of the table.
  • tableAll_rows_selected : Datatable object that collects the rowid from the selected table. This is then used to gather the GEOID from the table. The GEOID also the app to subset the envoData dataset and produce a new spatial object that is rendered on the map.
  • mymap_shape_click$id : The id parameter from the leaflet object also the quick indexing of the GEOID. This allows the sorting of the datatable object.

Reactive Elements

tabset panels Not really a reactive element, nothing about these table changes besides which feature is shown, but from the user prespective they seem like on. These tabset panels play an important role in the application because that provide all the contentual information people need to understand what the applicaiton is talking about. That said, I don’t like them. They are oddly finick in weird ways. They also don’t see as adaptable as most other shiny elements. For example, I can create a in applicaiton link to the map by defining a CSS class object for the fluid row that it rests in. I can’t do this for the tabset panel element. It doesn’t seem to respect the CSS structure in the same why the fluidrows do. I don’t have a valid explanation. This rant is here to point caution to this useful but not very robest element of the application.

Map Selection This series of buttons defines the behavior or the map, histograms, and table elements of the applicaiton. Changes to the map require use of the updata map button. Where as changes to the histogram and tables are reactive to changes in the geo scale selection. Odd behavior, from the users persepetive, can result when the geography displayed on the map does not match the geography current selected on under the geo scale drop down. Basically the GEOID from map selections will never match the content in the histogram and tables so there is a lost of functionality. The remove highlighted features from the map works indepently of other features. It’s kinda out of place from a functional perspective, but works well for user interaction.

Map This beauty is map possible because of the rmapshaper::ms_simplify() function which greatly reduces the complexity of input spatial datasets (up to and order of magniture in overall file size) while maining topological relationships. We are apply the function within the data processing code. The map is effect by four specific actions

  1. The map is initial created with a predefined subset of the data. This allows for quick loading and allows the handling of all addational map layers.
  2. As the user interacts with the map selection buttons the map is updated using a proxy call on the existing object. This allows user to change what is presented as the Indicator map layer.
  3. Adding highlighted feature on the table to the map all relies on a proxy call.
  4. Removing highlighted features from the map utilizes a proxy call.

Histograms The histograms are all rendered about a single function. No proxy elements are used, meaning everything a new map feature is selected or a new geo scale is select all 6 histograms are regenerated. Labeling is handled with clauses within the function. The visualization effect for highlight the bin required some duplication of code with the if/else statement. Should be obvious within the function, I just found that I ofter only edited one element rather then both when troubleshooting. Last note is than running locally the histograms seem to be effected by the size of the plot display within your rstudio environment. This is very odd, but if you end up with a ‘figure margin error’ then it’s worth increasing the display size of the plot viewer pane before looking to the code. I never encountered this error with deployment to shinyapps.io

Data Tables There is nothing to special about the data table themselves. They are indexed from a single datatable that is a subset of the current selected geography. There are a lot of changes to make here if you end up changing indicator names. As noted above the Indicator Description table breaks the mold of the other tables and does not react the same way as others. It be great to improve this at some point but for not I think it is a reasonable defect to maintain.

Functions

Functions are used within the application for one of two reasons.

  1. They represent a reusable code chunk that is deployed multiple times
  2. They represent a container for an isolated code chuck so that lines of code can be removed from the app.r file

My interest in utilizing function for contains has a cost when it comes to indicator name changes. You have to replace features names within the the app.r and functions. Still the endless scrolling of hunderd of line of code is bothersome so we’ve got container functions.

appPolyLine

description: Adds a polyline feature to an existing map object inputs :

  • map : leaflet map object this
  • sf1 : spatial data representing the polyline note: polyline rather then polygon due to the visualization technique apply to these features.
  • group : layerId, used to turn feature on and off in the map
  • popup : Text description to include in the popup outputs : returns a visualed polyline object that can appear on the map. considerations: Created to add the three community layers (oil, coal, rural). Mostly a container function.

createMap

description: This defines the inital map object. Lots of leaflet functions coming in to make the map. inputs

  • mapData : sf object used to define the indicator layer
  • pal : vector of hex codes used for the legend of the indicator layer
  • palMap : leaflet pallette object used to visualize the indicator layer
  • diPal : leaflet pallette object used to visualize the DI layer
  • oil : sf object for the oil community layer
  • rural : sf object for the rural community layer
  • coal : sf object for the coal community layer
  • di : sf object for the di community layer
  • justice40 : sf object for the justice40 layer
  • storyMaps : sf object for the story maps layer outputs : leaflet map object considerations: Mostly a container function. Calling on other functions within. Keep care to ensure the group names remain consistent as they occur in multiple places within the function.

defineLegend

description: Assigns label values to legend based on indicator name. inputs

  • indicator : indicator name. outputs : vector of character values to be used to label indicator layers within the legend
    considerations: If you change indicator names in the input datasets you have to change them within this function as well.

genPlots

description: Comprehensive function for generating the histograms. inputs

  • dataframe : envoData filtered to geography of interest
  • parameter : Component score to be visualized
  • geometry : Character from geo scale drop down, used to determine axis labels.
  • geoid = NULL : value from user map click. Set to null to default. If statement built around this parameter which determines if a single color are two colored histogram is generated. outputs : plotly object
    considerations: There are a lot of conditional statements in here as this is applied to 6 different indicators at three different geographies. Be aware of the if statement around the geiod parameter as there is a lot of duplicated code present there.

genTable

description: Largely a multi part select funtion to produce the table the user wants . inputs

  • tableData : envoData filtered to geography of interest
  • geoid : Value from the selection of a map feature. Used to order the table data
  • colselected : user input selection of what table to view.
    outputs : datatable object
    considerations: This function does not include the Indicator Description table. Any changes to names in the input dataset will require alterations here.

getDI

description: Container function to contrust the DI community map layer . inputs : NA outputs : SF object
considerations: All this can be done within the data processing code base, consider moving it out of the application in the future.

getStoryMaps

description: Generate the story map sf object. inputs : NA outputs : SF object
considerations: We only use this story may data within the shiny app. Will need to update the function once we get more links.

initialMapData

description: Simply container function to simplify input dataset for intial map construction. inputs : SF object outputs : SF object
considerations: NA

SCSS and inline HTML

Styling within the shiny application occurs in three different ways.

  1. Through the bslib::bs_theme function.
  2. Through class definations and scss script
  3. Through inline html/css

The base theme is set by the bslib fuction. Changing these parameters will directly alter the look of the application. That said this library is mostly responsabile for the look of the buttons.

Within the bslib funciton a scss file is reference as a source of addational rules. The style.scss file was generated incrementally to address specific visualization requests. Rules within in the scss file are ment to override generic theme set by the bs_theme. Rules were attempted to be made as specifcally as possible, thought there are some generic applications such as styling all p() calls.

Inline html/css was used as sparingly as possible and over time it would be ideal to phase it out entirely. For example we utilize inline html to set the in application relative link and maintain the proportion of the map element.

# display map -------------------------------------------------------------
fluidRow(tags$style(type = "text/css", "#mymap {height: calc(100vh - 250px) !important;}"), #style = {"background-color:#4d3a7d;"},
         column(1),
         column(7, leafletOutput("mymap")),
         column(3, br(),br(),br(),br(),
                plotlyOutput("histEnviroScreen" ,height = "80%", width = "100%")),
         column(1),

),

Once you get the hand of utilizing a code editor within your browser, this can be pretty fun stuff to experiment with.

Adaptations for Spanish

Ay Yai Yai, multilingual applications.

There is tremedous room for growth here. The intital development of the spanish language version of enviroscreen took place in the last month of the project. The implimention of application at release gets negitive points for elegance and parsimony. Why all the trouble.

Special Characters, sentence structure, and variable prepositions

special characters

Special characters have cause issues. Saving the files with encoding, “utf-8”, has resolved the majority of these issues. The persistent problem is that a character string with special characters can not be passed through a function. The work around for this was to dissolve all functions into the app.r file. As most functions were simply containers for saving space this did not effect the functionality of the application. That said it does mean there are numerous redundent code chunks.

Special characters have also cause problems in the input RDS file. They were not being save connectly. The work around here was utilize the dplyr::select function to rename all columns in the input dataset to ensure utf-8 formating.

The benifit of both these alterations is that all occurences of input parameter names are present in the app.r script. This means adapting the code to account for changes in indicator names is significantly more straighforward as compared to the english implimentation.

Sentence structure Being a different language, Spanish utilizes a different sentence structure then English. This effected the position of intext formating and hyperlinks. Once dealt with this is not a major concern but it is noted here because any addational translation efforts due require addational time for implementation.

variable prepositions “Puntaje de Colorado EnviroScreen” “Percentil del puntaje de Colorado EnviroScreen” “Contaminación y carga climática” “Percentil de contaminación y carga climática”

Sometimes it’s del sometimes it’s de. Nothing wrong with either. The challenge came from that fact that the application contrusts the percentil indicator names based on the input variable name alone. That is we don’t store the percentile del puntaje de Coloroado Enviroscreen in a drop down list. Instead we store “Puntaje de Colorado EnviroScreen” and measure or percentile. Based on the selection the indicator name is selected. This resulted in an addational clause statement. Not that big of a deal but this is a potential problem others will need to address in the future.

# indicator
       in1 <- input$Indicator
       # need to lower the first letter in the string but nothing else
       # grab first letter and set to lower
       t1 <- tolower(str_sub(in1, 1, 1))
       # subset out first letter
       t2 <- str_sub(in1, 2, nchar(in1))
       # construct new string
       indicator1 <- in1
       # sometimes it's del sometimes it de
       if(in1 %in% c("Puntaje de Colorado EnviroScreen"
                     ,"Indicador de descargas de aguas residuales"
                     ,"Indicador de salud mental"
                     ,"Porcentaje de discapacidades"
                     ,'Porcentaje que no completaron los estudios de secundaria'
                     ,"Porcentaje de aislamiento lingüístico"
                     ,"Porcentaje de bajos ingresos"
                     ,"Porcentaje de personas de color"
       )){
         indicator2 <- paste0("Percentil del ", t1,t2)
       }else{
         indicator2 <- paste0("Percentil de ", t1,t2)
       }

       if(input$Percentile == "Valor medido"){
         indicator <- indicator1
       }
       if(input$Percentile == "Rango percentil"){
         indicator <- indicator2
       }

Spanish/English Overall

Translation work presents a unique set of challenges. The benefit of the process is that it forces you to reevaluate the structure and function of your application. The result in the case of EnviroScreen two applications that take very different routes to get to the same place.

Questions

carverd@colostate.edu