In this second part of the RchivalTag tutorial series, we will learn how to read, analyze and visualize depth and temperature time series data.
To run this tutorial we will need RchivalTag version >= 0.1.5 and oceanmap version >= 0.13. You can find the newest versions of both packages on github
## install or load package
# install.packages("RchivalTag") # from CRAN
# library(xfun)
# install_github("rkbauer/oceanmap") # newest version from github
# install_github("rkbauer/RchivalTag") # newest version from github
library("RchivalTag")
## Package overview and version:
?RchivalTag
help(package="RchivalTag") ## list of functions
Recovered Archival Tags provide researchers with detailed information about the vertical behavior of fish and other aquatic animals such as turtles and mammals. Pop-Up archival tags can also transmit (subsets of) such data (e.g. miniPATs from Wildlife Computers) to the ARGOS satellite system, and thus do not need to be recovered. We therefore distinguish between high resolution time series data (3-10s sampling resolution from recovered tags, e.g. ###-Archive.csv files from recovered miniPATs, mk10, mk9 etc.) and and low resolution time series data (150-600s sampling resolution from transmitted data sets, e.g. ###-Series.csv files from miniPATs). Time Series data is incredibly powerful and may include data from different sensors (depth, temperature, acceleration, etc.). Here we will focus on Depth Time Series data. To read this data, you might be tempted to read in the data via R’s read.table or read.csv functions:
## read example transmitted Depth Temperature Time Series data:
ts_file <- system.file("example_files/104659-Series.csv",package="RchivalTag")
ts_df <- read.csv(ts_file)
str(ts_df,1)
'data.frame': 4163 obs. of 14 variables:
$ DeployID : Factor w/ 1 level "15P1019": 1 1 1 1 1 1 1 1 1 1 ...
$ Ptt : int 104659 104659 104659 104659 104659 104659 104659 104659 104659 104659 ...
$ DepthSensor : logi NA NA NA NA NA NA ...
$ Source : Factor w/ 1 level "Transmission": 1 1 1 1 1 1 1 1 1 1 ...
$ Instr : Factor w/ 1 level "MiniPAT": 1 1 1 1 1 1 1 1 1 1 ...
$ Day : Factor w/ 15 levels "07-Aug-2016",..: 1 1 1 1 1 1 1 1 1 1 ...
$ Time : Factor w/ 288 levels "00:00:00","00:05:00",..: 99 100 101 102 103 104 105 106 107 108 ...
$ LocationQuality: logi NA NA NA NA NA NA ...
$ Latitude : logi NA NA NA NA NA NA ...
$ Longitude : logi NA NA NA NA NA NA ...
$ Depth : num 105 85 72 23 46 16.5 36 23 23 23 ...
$ DRange : num 14 7.25 7.25 3.5 7 3.5 3.75 3.5 3.5 3.5 ...
$ Temperature : num 14.2 14.2 14.2 16.3 15.1 18 15.5 18 18 17.6 ...
$ TRange : num 0.25 0.25 0.25 0.25 0.25 0.25 0.25 0.25 0.25 0.25 ...
head(ts_df,3)
DeployID Ptt DepthSensor Source Instr Day
1 15P1019 104659 NA Transmission MiniPAT 07-Aug-2016
2 15P1019 104659 NA Transmission MiniPAT 07-Aug-2016
3 15P1019 104659 NA Transmission MiniPAT 07-Aug-2016
Time LocationQuality Latitude Longitude Depth DRange
1 08:10:00 NA NA NA 105 14.00
2 08:15:00 NA NA NA 85 7.25
3 08:20:00 NA NA NA 72 7.25
Temperature TRange
1 14.2 0.25
2 14.2 0.25
3 14.2 0.25
This works out perfectly, but I do not encourage you to do so since some transmitted data sets have gaps. Such gaps are not indicated in the transmitted data sets. In addition, we should convert the Day and Time columns to a datetime format so that we can plot and analyze our data with ease. Fortunately, RchivalTag
has it’s own function to read time series data from different manufacturers (e.g. Wildlife Computers and LOTEK) that fills gaps with NAs
and does the datetime conversion automatically.
## read example transmitted Depth Temperature Time Series data via RchivalTag:
ts_file <- system.file("example_files/104659-Series.csv",package="RchivalTag")
ts_df <- read_TS(ts_file)
str(ts_df,1)
'data.frame': 4320 obs. of 11 variables:
$ DeployID : chr "15P1019" "15P1019" "15P1019" "15P1019" ...
$ Ptt : int 104659 104659 104659 104659 104659 104659 104659 104659 104659 104659 ...
$ date : Date, format: "2016-08-07" ...
$ datetime : POSIXct, format: "2016-08-07 00:00:00" ...
$ DepthSensor: logi NA NA NA NA NA NA ...
$ Source : chr NA NA NA NA ...
$ Instr : chr NA NA NA NA ...
$ Depth : num NA NA NA NA NA NA NA NA NA NA ...
$ DRange : num NA NA NA NA NA NA NA NA NA NA ...
$ Temperature: num NA NA NA NA NA NA NA NA NA NA ...
$ TRange : num NA NA NA NA NA NA NA NA NA NA ...
We can see that the time series data has been converted. We have a date and datetime column in Date and POSIXct format (instead of the unformatted Day and Time columns before). In addition, the dataframe now starts with the “2016-08-07 00:00:00” while before it was “2016-08-07 08:10:00”, so the gaps have been completed even those before the tags deployment. This is an important step, also for other conversions (e.g. to histogram data) that we will discuss later.
I would like to further highlight the advanced date time conversion capabilities of RchivalTag
since this can be a tricky problem (e.g. when the data is in a time and or local language format). For example, this is the same data but with another date time format. Actually, the time format in the different tagging data files often differs (e.g. in case of Wildlife Computers), so the next code is actually pretty interesting.
Let’s try to read the same data in another time format.
## other date_format:
ts_file2 <- system.file("example_files/104659-Series_date_format2.csv",package="RchivalTag")
ts_miniPAT2 <- try(read_TS(ts_file2)) # run to see error message
Error in .fact2datetime(ts_df$datetime_tmp, date_format = date_format, :
date concersion failed! Please revise current'date_format': %d-%b-%Y %H:%M:%S to 07-08-2016 08:10:00
So RchivalTag
tells us what to do. We need to adjust the standard date format to ‘07-08-2016 08:10:00’. Let’s do that:
## changing the date format and
ts_miniPAT2 <- read_TS(ts_file2, date_format = "%d-%m-%Y %H:%M:%S")
str(ts_miniPAT2,1)
'data.frame': 4320 obs. of 11 variables:
$ DeployID : chr "15P1019" "15P1019" "15P1019" "15P1019" ...
$ Ptt : int 104659 104659 104659 104659 104659 104659 104659 104659 104659 104659 ...
$ date : Date, format: "2016-08-07" ...
$ datetime : POSIXct, format: "2016-08-07 00:00:00" ...
$ DepthSensor: logi NA NA NA NA NA NA ...
$ Source : chr NA NA NA NA ...
$ Instr : chr NA NA NA NA ...
$ Depth : num NA NA NA NA NA NA NA NA NA NA ...
$ DRange : num NA NA NA NA NA NA NA NA NA NA ...
$ Temperature: num NA NA NA NA NA NA NA NA NA NA ...
$ TRange : num NA NA NA NA NA NA NA NA NA NA ...
This worked out! However, sometimes the data is formatted in another language format as indicated earlier:
## other language format and separator:
ts_file_ES <- system.file("example_files/104659-Series_date_format_ES.csv",package="RchivalTag") # spanish date format
ts_miniPAT_ES <- try(read_TS(ts_file_ES)) # run to see error message
WC.miniPAT.data.with.spanish.date.format
1 DeployID;Ptt;DepthSensor;Source;Instr;Day;Time;LocationQuality;Latitude;Longitude;Depth;DRange;Temperature;TRange
2 15P1019;104659;;Transmission;MiniPAT;07/abr/16;08:10:00;;;;105;14;14.2;0.25
3 15P1019;104659;;Transmission;MiniPAT;07/abr/16;08:15:00;;;;85;7.25;14.2;0.25
4 15P1019;104659;;Transmission;MiniPAT;07/abr/16;08:20:00;;;;72;7.25;14.2;0.25
5 15P1019;104659;;Transmission;MiniPAT;07/abr/16;08:25:00;;;;23;3.5;16.3;0.25
6 15P1019;104659;;Transmission;MiniPAT;07/ene/16;08:30:00;;;;46;7;15.1;0.25
Error in `$<-.data.frame`(`*tmp*`, "datetime_nm", value = numeric(0)) :
replacement has 0 rows, data has 11
In this case we need to change the separator, the date and language format to ‘07/ene/16;08:30:00’. Turns out this is a spanish format.
ts_miniPAT_ES <- read_TS(ts_file_ES,skip=1,sep=";",header = TRUE,
date_format = "%d/%b/%y %H:%M:%S",lang_format = "es") # spanish date format
head(ts_miniPAT_ES,2)
DeployID Ptt date datetime DepthSensor Source
1 15P1019 104659 2016-01-07 2016-01-07 00:00:00 NA <NA>
2 15P1019 104659 2016-01-07 2016-01-07 00:05:00 NA <NA>
Instr Depth DRange Temperature TRange
1 <NA> NA NA NA NA
2 <NA> NA NA NA NA
As mentioned earlier, read_TS
also works with data from other manufacturers (currently only LOTEK). Here is the same data in the LOTEK PSAT Dive format:
## load same data in LOTEK format
ts_file <- system.file("example_files/104659_PSAT_Dive_Log.csv",package="RchivalTag")
ts_df <- read_TS(ts_file,date_format="%m/%d/%Y %H:%M:%S")
head(ts_df,2) ## attention no identifier (Ptt, Serial, DeployID) included!
DeployID Ptt date datetime Validity Temperature
1 NA NA 2016-08-07 2016-08-07 00:00:00 NA NA
2 NA NA 2016-08-07 2016-08-07 00:05:00 NA NA
Depth
1 NA
2 NA
This worked out, however, LOTEK does not include the DeployID and Ptt information in their time series data. We therefore need to add this information manually. This will be important when further converting and visualizing our data.
## add missing DeployID and PTT information to LOTEK depth temperature time series data:
ts_df$DeployID <- ts_df$Ptt <- "104659"
Now let’s plot this data. RchivalTag
has several plotting functions for time series data. These include:
hist_tad
ggboxplot_DepthTS_by_hour
plot_DepthTS
empty.plot_TS
plot_DepthTempTS
plot_DepthTempTS_resampled
plot_DepthTempTS_resampled_PDT
dy_DepthTS
In the first part of this tutorial we have seen how to read and plot histogram data, which can also be obtained from depth time series (DepthTS) data. Although histogram data is an efficient data product to provide an overall overview of general depth layer preferences, its value to analyze diel vertical behavior is limited. On the other side, analyzing DepthTS data from one ore more individual can be tricky, because of the duration and variability in these data sets. Still there are some handy functions in RchivalTag that can help to perform this job, that we will learn in a sec. For now, let’s say we are interested in getting a general idea of the diel vertical behavior depth preferences (which is a good starting point of a more in-depth analysis of DepthTS data). For this purpose, RchivalTag includes the function ggboxplot_DepthTS_by_hour
.
ts_file <- system.file("example_files/104659-Series.csv",package="RchivalTag")
ts_df <- read_TS(ts_file)
ggboxplot_DepthTS_by_hour(ts_df)
Let’s add position data to obtain twilight and nighttime shadings. We could add here fixed position values.
ts_df$Lon <- 5; ts_df$Lat <- 43
ts_df2 <- get_DayTimeLimits(ts_df)
ggboxplot_DepthTS_by_hour(ts_df2,ylim=c(0,100))
However, it is more accurate to use the maximum likelihood positions from the GPE3 model runs, especially if the animal is highly migratory and thus moving over large distances. This is particularly important when we analyze the raw depth time series data. By contrast, the ggboxplot_DepthTS_by_hour
function will calculate the average timing of sunrise and sunset.
We can read in the maximum likelihood positions via the get_geopos
function. We will learn more about this function and the visualizing of the tracks in the 4th part of this tutorial.
library(dplyr)
gpe3_file <- system.file("example_files/15P1019-104659-1-GPE3.csv",package="RchivalTag")
tracks <- get_geopos(gpe3_file)
Loading tagging tracks from file: /home/work/R/x86_64-pc-linux-gnu-library/3.6/RchivalTag/example_files/15P1019-104659-1-GPE3.csv
add <- tracks %>% select(DeployID,Ptt,Lat,Lon,date) %>%
group_by(DeployID,Ptt,date) %>%
summarise(Lat=mean(Lat),Lon=mean(Lon)) %>%
mutate(datetime=RchivalTag:::.date2datetime(date))
add <- classify_DayTime(add) %>% select(-datetime)
dat <- ts_df %>% select(-Lon,-Lat)
ts_df2 <- add %>% select(-DeployID) %>% inner_join(dat,by=c("date","Ptt","DeployID"))
ggboxplot_DepthTS_by_hour(ts_df2,ylim=c(0,100))
Now let’s checkout the underlying, raw depth time series data. We can directly plot the entire data set with the plot_DepthTS
function:
plot_DepthTS(ts_df)
This function has a bunch of arguments that we will go through. Some basic arguments like xlim
to select the dates to be plotted:
plot_DepthTS(ts_df,xlim = "2016-08-10")
Another way to produce this figure is via the empty.plotTS
function that will draw an empty plot first and then gives us a line.
# example for empty.plotTS and adding time series data as line:
empty.plot_TS(xlim="2016-08-10",ylim=c(100,0))
lines(ts_df$datetime, ts_df$Depth)
Many aquatic animals show diurnal migration patterns so we might be interested in shading the night-time and twilight periods. To do so we need to add at least the Longitude and Latitude information since the sunrise and sunset timings change differ with location. Let’s do this manually.
# plot also day night time information:
ts_df$Lon <- 5; ts_df$Lat <- 43 # manual example, please take Lon/Lat data from (GPE3) model outputs for your analysis instead. (check get_geopos-function from RchivalTag)
ts_df <- classify_DayTime(ts_df)
plot_DepthTS(ts_df,xlim = c("2016-08-10","2016-08-15"),plot_DayTimePeriods = T)
We can do the same figure with an empty plot and the vertical track as line on top:
# alternative:
plot_DepthTS(ts_df, xlim=c("2016-08-10","2016-08-12"), plot_DayTimePeriods = TRUE, type='n')
lines(ts_df$datetime, ts_df$Depth)
The dy_DepthTS
function allows us to to the same figures but with the interactive interface of the dygraphs
package.
ts_file <- system.file("example_files/104659-Series.csv",package="RchivalTag")
ts_df <- read_TS(ts_file)
ts_df$Serial <- ts_df$DeployID
dy_DepthTS(ts_df)
Adding night and twilight shadings:
ts_df$Lon <- 5; ts_df$Lat <- 43
dy_DepthTS(ts_df)
Some further arguments:
dy_DepthTS(ts_df, xlim = unique(ts_df$date)[2:3], plot_DayTimePeriods = FALSE, doRangeSelector= FALSE)
Remove grid, plot points:
library(dygraphs)
dg <- dy_DepthTS(ts_df, xlim = unique(ts_df$date)[2:3], plot_DayTimePeriods = FALSE, drawPoints = TRUE)
dg <- dyOptions(dg,drawGrid=FALSE)
dg
RchivalTag
fills gaps with NAs. However, both plot_DepthTS
and dy_DepthTS
function can also deal with data data gaps:
ts_gaps <- ts_df
ts_gaps$Depth[c(300:800)] <- NA
ts_cut <- ts_df[-c(300:800),]
plot_DepthTS(ts_gaps)
plot_DepthTS(ts_cut) # same result
dy_DepthTS(ts_gaps) ## same figure with dy_DepthTS
dy_DepthTS(ts_cut) # same result
plot_DepthTS
is a base
-graph. So it comes with almost all the functionalities of the base plots. We could for example also add another line and axis for the temperature data.
par(mar=c(5,4,4,5)) ## change margins to add second axis label
ts_sub <- plot_DepthTS(ts_df,xlim = c("2016-08-10","2016-08-15"),plot_DayTimePeriods = T, Return=T)
par(new=T) ## plot on top of first depth time series plot
ylim <- range(pretty(range(ts_sub$Temperature)))
plot(ts_sub$datetime, ts_sub$Temperature, xlim=range(ts_sub$datetime), ylim = ylim, type='n',axes=F,xlab="",ylab="") # create empty plot
lines(ts_sub$datetime, ts_sub$Temperature,col="orange") # add line
axis(4, at = pretty(ylim)) # add axis
mtext(side=4,"Temperature (°C)",las=0,line=2.3) # add axis label
Let’s create a second line for the temperature time series data with dy_DepthTS
and dygraphs
:
dg <- dy_DepthTS(ts_df) # run via dy_DepthTS to get shadings
## manually create the same figure with a second line
dat <- ts_df[,c("datetime","Depth","Temperature")]
dat$datetime <- as.POSIXct(dat$datetime,tz = "UTC")
dat_xts <- xts::xts(dat[,2:3],order.by=dat$datetime)
dg2 <- dygraph(dat_xts) # create initial dygraph object
dg2$x$shadings <- dg$x$shadings ## assign shadings
# define limits of y-axes
ylim1 <- c(max(ts_df$Depth,na.rm=T),-1)
ylim2 <- c(12,30)
dg2 <- dg2 %>% dyRangeSelector() %>%
dySeries("Temperature", axis = "y2") %>% ## set up second y-axis
dyOptions(labelsUTC = TRUE, strokeWidth = 1) %>% ## set datetime format to UTC (default is local time zone)
dyAxis("y", label = "Depth", valueRange = ylim1) %>% ## set limits and label of y1-axis
dyAxis("y2", label = "Temperature", valueRange = ylim2) ## set limits and label of y2-axis
dg2
Well this worked, but it was quite tricky and it’s not that pretty either. RchivalTag
has some additional functions to illustrate depth-temperature time series data in a much nicer way. One option is plot_DepthTempTS
. In this case, adjacent depth-temperature records will be linearly interpolated. The outcome looks good, but is not recommended for low resolution time series data (>= 300 s sampling resolution), because the potential interpolation between distant records might be misleading (e.g. 30 degrees at the surface and 10 degrees at 400m leads to a false temperature profile, with 20 degrees at around 200 m)… .
Another option is to interpolate the temperature based on daily resamples (average profiles) via the plot_DepthTempTS_resampled
. Both of these functions are adaptations of plot_DepthTS
, which means we have similar arguments (e.g. to produce night-time and twilight shadings).
plot_DepthTempTS_resampled(ts_df,xlim = c("2016-08-10","2016-08-15"),plot_DayTimePeriods = T,tz="HST")
Another option is to estimate the temperature profiles from external data sets. PDT-data (PAT-Style Depth-Temperature profiles) from Wildlife Computers are one example for that.
ts_df$Temperature <- c()
pdt_file <- system.file("example_files/104659-PDTs.csv",package="RchivalTag")
PDT <- read_PDT(pdt_file)
running pdt_file /home/work/R/x86_64-pc-linux-gnu-library/3.6/RchivalTag/example_files/104659-PDTs.csv
plot_DepthTempTS_resampled_PDT(ts_df,PDT,xlim = c("2016-08-10","2016-08-15"),plot_DayTimePeriods = T,tz="HST",do_interp = F)
We will discuss this the different types of temperature profiles in the 3rd part of this tutorial.
For attribution, please cite this work as
Bauer (2020, Nov. 20). Marine Biologging & Data Science | Blog: RchivalTag Tutorial | Part 2 (Interactive) Time Series Data Plots. Retrieved from http://oceantags.com/posts/RchivalTag_Tutorials_Part2_Time_Series_Data/
BibTeX citation
@misc{bauer2020rchivaltag, author = {Bauer, Robert K.}, title = {Marine Biologging & Data Science | Blog: RchivalTag Tutorial | Part 2 (Interactive) Time Series Data Plots}, url = {http://oceantags.com/posts/RchivalTag_Tutorials_Part2_Time_Series_Data/}, year = {2020} }