Methods for digitizing contour plots, adding Lidar data, and calculating DEM’s of difference

Note: If something doesn’t work. Save Arc. Close arc. Open arc. Magic!

I.   Scanning Alger Contour plots into ARC

a. Adding map document to Arc found at http://courses.umass.edu/nrc592g-cschweik/pdfs/Georeferencing_a_map.pdf

b. Copy image (Contour plot from Alger) into paint.net and save as XX_Alger.jpg

c. Start ArcMap

i.  Customize è extensions è check “3d Analyst, ArcScan, and Spatial Analys

ii.    Customize è Toolbars è Add ArcScan, Georeferencing , and Editor

iii.    File è Map document properties è check “Store relative pathnames to data sources”

iv.   Catalog è Connect to folder è “Thesis”

d. Setting jpg properties and adding to map

            i.      Folder Connections è .jpg file è right click è properties è Spatial Reference è Edit è WGS 1984 UTM 58 S è Appl

           ii.      Add Data è jpg file è Build pyramids (Yes)

e.  Georeferencing Contour map

 i.   Click “Add control points” on georeferencing toolbar

 ii.   Zoom in to BM1

 iii.   Click once on BM1 then again to turn crosshair red from green

 iv.   Repeat for all BM’s

  v.   Once all BM’s are marked, open excel file “1993_Survey_All_Transects” and note lat/long of all BM’s

 vi.   Open “Link Table”(Sometimes located just off edge of screen…) è Replace X_Map and Y_Map with BM’s lat/long è Zoom to full extent to go to true map location

  vii.  In georeferencing toolbar, click “Rectify” è Change save location to proper folder è Save saves as XX_Alger1.tiff file with “1” added to end of name

 viii.  Close Map Doc è start blank map è Click “+” (Expand tiff file) button to new tiff file è Add 1 layer (Band_1) of rectified tiff to new map

   ix.  In Table of Contents, right click Tiff file è Properties è Symbology è Classified è Classes = “2” è “OK”

II.  Creating Vector data from contour lines in Alger Scans

a.   Information found at http://www.fas.harvard.edu/~chinaarch/Downloads/gis/L05_creatingvector/L05_creatingvector%20English.pdf

b. Creating a new shapefile in arc catalog

        i.   In Catalog, right click folder that contour plot is in è New è Shapefile

            1.   Name with NO SPACES similar to “XX_Shp.shp”

            2.   Feature Type = “PolyLine”

           3.   Coordinate System = “58 S”

           4.   “OK” 

     ii.   Adding Attributes 

          1.    Right Click new shapefile in Catalog! è Properties è “Fields” Tab è Below ID in Field Name type “Elevation” è under Data Type select “Double” è “OK”

c.  Creating Vectors

    i.   In Editing Toolbar select “Start Editing”, Then select the Shapefile in “Create Features”, and then “line”

   ii.    In ArcScan toolbar select “Raster Snapping Options” button è Check for correct values (What are they? 20,5,200,0) è “OK” Make sure arcscan is up and other features to make vectorization work 

   iii.   Under the “Vectorization” Menu, select “Vectorization Settings” è “Styles” è “Contours” è “OK” 

   iv.   Next to the Vectorization Menu, select “Vectorization Trace” Tool.

   v.    Click on beginning of Contour Line, and allow arrow to trace line automatically 

          1.   Sometimes at intersections or other unnatural features, the line stops. “Jump” Over the break with “s” and continue on with line

           2.   Double Click or press F2 to end Vector trace

          3.    Arc loves to not work after crossing a line. It jumps giant gaps to connect with completely unrelated lines although it should follow the line indicated. Need to contact Arc about this, or just finish the contour manually.

           4.    If there are large gaps in the contour lines, manually change the classification scheme to account for gray areas to be black and not white.

      vi.  ArcScan allows for automation of contours, but I don’t believe this works very well. Instructions in website or pdf above

    vii.   Once line is created, go into Attribute Table of shapefile and add elevation of contour line just created

           1.       Check BM elevation from Alger

           2.       Check BM elevation from UNAVCO

           3.       +/- difference from the two for each contour elevation

    viii.      Repeat for all contour lines

      ix.      “Stop Editing

       x.      Save Map Frequently as ex. “XX.mxd”

III.                Topo to raster 3D analyst for now? Spatial does stream polyline better(Not using this anymore)

a.       Input feature data = XX_Shp.shp

b.      Field = Elevation

c.       Rename and save to proper folder! “XX_T2R”

d.      Cell size=? Make everything 0.1 (meters) but takes forever with Lidar

e.      After finishing, can change the symbology to full 32 layers instead of ~9.

  i. Formatting

      1. Need to figure out how to label Orange, Green, and Black mats so can tell differences between mats and seasons.

f.  Water Surface Elevations. Maybe?

g.  Rocks

h.  Other important features

IV.   Cropping out extra raster (Not using this either anymore)

a.  Need to crop out extra raster data because is not correct or accurate

    i.  Make new shapefile (XX_Clp.shp)

1.   Polygon, and same coordinate system

     ii.      Start Editing, select XX_Clp

     iii.      Trace perimeter of contour shapefile. Make sure end snapping is selected.

     iv.      Clip(Data Management)

1.   Select Raster to clip è XX_T2R

2.   Output extent è XX_Clp

3.   Select “Use input features…”

4.   Rename output è XX_T2R_Clp

5.   May need to change symbology from stretched to classified. This asks to compute histogram. Select yes. Then import data from XX_T2R for same color scheme.

V. Importing LiDAR data into ARC

a.   If Lidar data is single file continue to b. Otherwise follow below.

    i.   Open up a new command window in dos. Click start and type cmd

   ii.   Navigate to folder containing lidar sets using the command “cd foldername” to go up one folder at a time or “cd..” to go back a folder. Ex. C:\Users\Crisp\Desktop\Thesis\Antarctic Scans\Lidar\F2

   iii.  Type “copy /b filename1 + filename2 + filename3 outfilename

    iv.    Concatenated file should appear in a few seconds!

b.  In Arc, Under “File” è Add data è Add XY data (This is very important XY data!!!)

   i.    Select LiDAR set

   ii.    Choose corresponding X, Y, and Z columns for data

   iii.   Update Coordinate system to 58S

   iv.   Hit OK

1. Once the addition is finished, it will bog down ARC adding each point. Unselect the Lidar data in the table of contents and proceed to point to raster below

c. Point to Raster in conversion tools

    i.   This is going to take a while so be prepared

    ii.   Choose newly imported XY data set

    iii.   Value field is Z or elevation

    iv.   Save file as XX_P2R_YYYY Unfortunately Arc is omnipotent and gets angry when there are more than 13 characters. Assumption is that cellsize (Below) is 0.1m

     v.  Cellsize = 0.1

1. This means each cell with be 0.1 m wide. You can change this to a smaller dimension, but it will take exponentially longer.

    vi.  You can get into detail with cell assignment and priority fields, but typically leave them at default

   vii.   Wait for 35-45 hours (min) for this to finish.

   viii.   Once completed, change symbology to from stretched to classified

1.       Symbology adoption however has problems.

2.       Ex. Lidar has range of 19-31 meters. Adopted contour plot has range of 25-28.

3.       Therefore everything on Lidar below 25 or above 28 is same color.

4.       Fix? Find the best fit scan/plot and correlate everything else to that piece

d.   Patching holes in raster and adding hillshade

    i.  “Raster Calculator” in Spatial Analyst

1.  Enter code è Con(IsNull("Input_Raster"),FocalStatistics("Input_Raster",NbrRectangle(3, 3),'MEAN'),"Input_Raster") è Output Raster = “XX_Patch” è OK

2. The input raster is the lidar DEM that has holes in it either due to water or shadows from rocks or poor TLS placement

3. The “3” is the width and height of the patching

4.  Explanation: The above code finds cells within the DEM that are NoData and takes a 3X3 matrix surrounding it, computes the mean, and then outputs that elevation value

     ii. “Hillshade” Spatial Analyst

1. Input Raster = XX_Patch è Output Raster = XX_YYYY_HS è azimuth altitude and z are optional to change è OK

2.  Need grayscale

3.  http://resources.arcgis.com/en/help/main/10.1/index.html#//009t0000007q000000

      iii.      Represent stream within channel at certain flows?

1.       ArcHydro

       iv.      Make a model builder!

VI.  Adding LAS data to Arc Scene

a.  LASTools

i.  Open Txt2Las in the lastools/bin folder

 ii.  Click Browse è \.. to go back folder. Search for RGB lidar file you want

 iii.   Output è Proper directory è Name file XX_YYYY_LAS

 iv. More attributes è Select R, G, and B

 v.  CHANGE FORMAT FROM LAZ TO LAS!!! Have no idea why this isn’t automatic

 vi.   Hit run! è takes ~30 seconds.

b.   Importing LAS file

  i.  Open new arcscene map and save as XX.sxd

  ii.  Navigate to folder being used for stream in folder connections

 iii.  Right click stream folder è new è LAS Dataset è name “XX.lasd”

  iv. Right click XX.lasd è Properties è XY coordinate system tab = 58S è LAS Files tab è Add files è select newly created las file from lastools è OK

   v. Drag XX.lasd into map

  vi.  In the LAS Dataset Layer toolbar select the symbolize button (Button with bunch of dots) and select RGB.

  vii.  Now have lidar map to fly through!

VII.  GCD – Geomorphic Change Detection

a.  Download and Install

      i. Download and install GCD 6 from http://gcd.joewheaton.org/downloads

    ii.   Go to Arc è Customize è Add in manager and GCD should automatically be added

    iii.    In toolbars select GCD and pin it to the toolbar

b.      Building DEMS

                                                               i.       

VIII.  Adding XY Algae Data to Alger Scans

a.  Algae Data

   i.   Formatting

1.    Into .csv’s

a.       Data should already be organized in one large spreadsheet based off of flow season

b.      Create 1 csv file for 1 stream for 1 season

                                                                                                                                       i.      Ex. Bowles_1213.csv, Bowles_1314.csv…

                                                                                                                                     ii.      .csv files should have 4 columns

1.       Easting

2.       Northing

3.       Orthometric height

4.       Description

                                                             ii.      Adding XY data

1.       Find .csv file è Create feature class è From XY table

2.       Change x and y and coordinate system

3.       Save as default name

4.       This creates a shapefile within the stream folder and not the default Arc folder which might be better for editing and finding etc.

5.       Add new shp file to arc map

b.      Stream Boundary

      i.      Create new polygon shapefile called “Stream_Boundary”

      ii.      Start editing

      iii.      Follow stream path and save edits

c.       Contour Polyline to point

  i.      Feature Vertices to Points

1.       Input Features = “XX_Shp”

2.       Output = “Contour_Points”

IX.                Point differencing from base lidar

a.       G = 1, B = 2, O = 3, R = 4

b.      Extract Values to points

   i.      Input point features “XYXX_YYYY”

   ii.      Input Raster “Patch”

1.       If not doing patch, the NoData cells indicate water. Can be used for algae analysis later on in different study

  iii.      Name “YYYY_E2P”

1.       YYYY = year

2.       E2P = Extract to points

   iv.      Interpolate points Check

c.       Clip(Analysis)

   i.      Input features = YYYY_E2P

  ii.      Clip Features = Stream_Boundary

  iii.      Output = YYYY_Clip

d.      Erase

    i.      Input features = YYYY_E2P

    ii.      Erase Features = Stream_Boundary

    iii.      Output = YYYY_Erase

e.      Export feature attribute to ASCII – Find in toolboxes not custom box

     i.      Input feature = YYYY_clip/erase

     ii.      Value field = Orthometri, RASTERVALU,  ( + Point_Desc for Clp file!)

    iii.      Delimiter = space

    iv.      Output = YYYY_Ers/Clp

     v.      Field names = Unchecked!

f.        Contour Clip and Erase

     i.      Perform same as above except export the contour season as 9293 so it plots next to 9394 or the season that it was created. Maybe figure a better way to plot this?