Mobile Sensing (full)

It is only possible to use mobile sensing when you have a mobile sensing premium account. This has to be purchased additionally to the normal premium account. Please contact m-Path for pricing (contact@m-path.io).

This article explains how to use mobile or passive sensing in m-Path. Full sensing essentially means that the sensing is continuous, so that the sensors are recording data even when the app is in the background. In mobile sensing lite, the sensors are only used during a questionnaire. The differences with mobile sensing lite are explained here:

The sensors

The available sensors are shown in the following table. However, it is not necessary to collect data from all these sensors. The researcher can choose which sensor data to collect. Some sensors are periodic (researcher can choose the period), while others are event based (new data entry is created in case of a specific event). The exact data output for each sensor is described here.

Type of data	Description	Android	iOS	Periodic or event
Location	GPS coordinates every minute.	✔	✔	Event for iOS (every 10 meter moved), periodic for Android
Device	Information about your device (e.g.: name, version, available working memory, battery level).	✔	✔	Periodic
Weather	Weather and air quality from the internet based on current location.	✔	✔	Periodic
Pedometer	Number of steps according to the pedometer of your mobile phone.	✔	✔	Periodic
Activity	Number of minutes walking, cycling, driving, etc.	✔	✔	Event (change of activity such as still or walking)
Screen	Time of screen on/off/unlocked events.	✔	❌	Event (with screen off or on).
Battery	Amount of battery left, if battery is charging or not.	✔	✔	Event (e.g. starting to start charging)
App usage	Usage time per app since the last measurement.	✔	❌	Periodic
Connectivity	Connected to Wi-Fi or data.	✔	✔	Event (e.g. connecting to Wi-Fi)
WIFI	Wi-Fi networks in the vicinity.	✔	✔	Periodic
Periodic accelerometer	Accelerometer (acceleration)	✔	✔	Periodic
Light	Light sensor, amount of light (in lux)	✔	❌	Periodic
Memory	The free physical memory and free virtual memory.	✔	❌	Periodic
Bluetooth	Information about Bluetooth devices in your area (e.g.: name, ID, signal strength)	(only when app in foreground, better to use Sense Lite)	(only when app in foreground, better to use Sense Lite)	Periodic
Ambient noise	Volume (in decibels) of ambient noise. No audio is saved.	Only during questionnaire (see Sense Lite)	Only during questionnaire (see Sense Lite)	n.a.

The m-Path Sense app

To use mobile sensing, participants need to download the m-Path Sense app in the App Store or in the Play Store. The difference with the normal app is that

1. m-Path Sense allows for using sensors in the background
2. m-Path Sense asks all permissions (with relation to the sensors) when the app starts for the first time

Android app

In the following video, the set-up of the Android app is shown. In the Android app, users will be able to see an m-Path Sense notification on top of the screen. This will inform them that the app is actually running. This notification however only works when location is gathered. It is therefore better to include location in the sensors.

iOS app

The set-up of the iOS app is very similar to the Android set-up, as shown in the images below. However, in iOS there will be no notification that will show that the app is running.

Keeping the app alive

If the app is closed down, the sensing will stop. Ideally, participants should never stop or close the app. If necessary, they can move the app to the background.

Both iOS and Android may kill or pause apps in an attempt to preserve battery. When the app is closed, the m-Path system will send a notification to the participant to remind the participant to reopen the app. There are multiple things a participant can do to prevent the pausing or killing of the app:

• Use the location sensor. Apps will be kept alive much longer and sensing will work better when the location sensor is used.
• Keeping the app in foreground
• Disabling battery restrictions on Android (on many phones this is automatically asked when starting the phone for the first time)
• Not ‘swiping away’ the app from the list of recently used apps
• Make sure participants don’t have too many other apps open (and have enough memory/battery)
• Follow specifics on https://dontkillmyapp.com/

In general it will not be possible to completely stop the phone from pausing or killing the app. Especially during the night, the amount of sensor data collected will decrease.

m-Path Sense settings

It possible to add a ‘Full mobile sensing setup’ question in any questionnaire. When a participant goes through this question, he or she will not notice anything, but in the background, the sensing settings will change. These sensing settings will be saved on the phone (until they are overwritten again). It is advised to add this ‘Full mobile sensing setup’ question in the intake questionnaire. This way, the sensing can start immediately when a participant has added a researcher.

In this setup (see figure below), it is possible to choose all the settings for mobile sensing. By clicking on ‘specific’, even more settings will be available.

• Study title: the title of the study. Shown in the name of export files.
• Description for participants: a small description for participants.
• Permanent notification title: the title of the notification on Android that shows that m-Path Sense is running.
• Permanent notification description: small description on Android notification.
• Permanent notification message: message on Android notification.
• Remind after app is closed (in minutes): m-Path Sense tries to connect with the m-Path servers every 5 minutes. If m-Path fails to do so (for example because the app is closed) for x minutes, m-Path will send a notification to the participant to restart the app. For example when this field is filled in with 30, 120, 360 m-Path will send a notification after 30 minutes, 120 minutes and 360 minutes of radio silence from the app.
• Reminder title: The title of the reminder notification
• Reminder message: The message of the reminder notification

It is also possible to choose the specific sensors in this m-Path sense setup editor. For all the sensors it is possible to

• Enable or disable the sensor.
• Choose the sensor interval. This interval sets the period for the sensor. However, this interval only holds for period sensors (not event based), as shown in the table above. This interval is in seconds.

In the underlying example figure, this means that only weather and activity is measured. Weather is measured every 1200 seconds (20 minutes). Activity is an event based sensor, a new data entry is added always when the phone detects a change in activity.

As mentioned above, Android and iOS will occasionally try to put the app in sleep mode ore try to stop the app. This means that the actual, average, measured interval is longer, and dependent on the phone (e.g. weather could on average be measured every 2000 seconds instead of the given 1200 seconds). It is however not a good idea to take too small sensor interval, as this will overburden the phone (shorter battery life and Android and iOS trying to shut down the app even more).

m-Path Sense data

All sensor data is immediately uploaded to the pCloud account of the researcher. This can account can be linked to m-Path as explained here:

The data is uploaded every 5 minutes if the participant has WIFI. If no WIFI is available, the data will only upload in case that there are 5 or more files to upload and there is at least cellular internet.

The data will be saved with names as 369_testStudy_192421_m_Path_sense_2023-10-28_14-28-52630128.json. The first number (in this case 369) is the number of the researcher. Then the name of the study is added. It is followed by the connectionId of the participant (in this case 192421). This connectionId can also be found in the export data of m-Path questionnaire answers. The filename ends with m_Path_sense and the date and time when the file was uploaded.

In the following paragraphs we will go over all sensors and their outputs.

Time zone

Time zone will always be included in the sensing data. It shows the current time zone of the participant.

{"sensorStartTime":1698493072161003,
"data":{"__type":"dk.cachet.carp.timezone","timezone":"Europe/Brussels"}},

Each sensor entry will always have a sensorStartTime. The sensorStartTime is in microseconds since Epoch. Using the time zone recorded in the time zone sensor this can be transformed to readable time. In R, this can be done with the following code:

t <- 1698493072161003;
format(as.POSIXct(t / 1000000, origin = "1970-01-01", tz = "Europe/Brussels"), "%Y-%m-%d %H:%M:%OS")

In this code, the microseconds since epoch was set to 1698493072161003 and the timezone was set to “Europe/Brussels”. This output in R is “2023-10-28 13:37:52”. This is the local time of the participant at which this sensor was recorded.

Note that timestamps in sensing data are somewhat different to the timestamps exported in the questionnaire data.

Location

Multiple features with regard to location and GPS are collected. Note that not all phones have access to all features. For example speed and accuracy is not recorded on some phones.

{"sensorStartTime":1698491782170268,
"data":{"__type":"dk.cachet.carp.location",
"latitude":51.12534824809812,
"longitude":4.484622647500936,
"altitude":4.432750540253859,
"accuracy":8.824313965098712,
"verticalAccuracy":1.4465603013884314,
"speed":2.2290189586659954,
"speedAccuracy":2.8313907994468623,
"heading":330.25365021341366,
"time":"2023-10-28T13:16:21.998",
"isMock":false}}

Device

Information about the device of the participant

{"sensorStartTime":1698493072149431,
"data":{"__type":"dk.cachet.carp.deviceinformation",
"platform":"iOS",
"deviceId":"55D78F8D-901B-4FA6-AD19-890D39578F99",
"hardware":"iPhone11,2",
"deviceName":"iPhone",
"deviceManufacturer":"Apple",
"deviceModel":"iPhone",
"operatingSystem":"iOS",
"deviceData":{"systemName":"iOS","isPhysicalDevice":"true","utsname":{"release":"22.6.0","sysname":"Darwin","nodename":"Merijns-iPhone","machine":"iPhone11,2","version":"Darwin Kernel Version 22.6.0: Wed Jun 28 20:51:42 PDT 2023; root:xnu-8796.142.1~1/RELEASE_ARM64_T8020"},
"model":"iPhone",
"localizedModel":"iPhone",
"systemVersion":"16.6",
"name":"iPhone",
"identifierForVendor":"55D78F8D-901B-4FA6-AD19-890D39578F99"}}},

Weather

Information about the current weather at the location of the participant.

{"sensorStartTime":1698493072496688,
"data":{"__type":"dk.cachet.carp.weather",
"country":"DK",
"areaName":"Virum",
"weatherMain":"Clouds",
"weatherDescription":"overcast clouds",
"date":"2023-10-28T13:37:52.000",
"sunrise":"2023-10-28T08:06:55.000",
"sunset":"2023-10-28T17:40:55.000",
"latitude":55.7944,
"longitude":12.4463,
"pressure":1005.0,
"windSpeed":4.86,
"windDegree":107.0,
"humidity":83.0,
"cloudiness":100.0,
"temperature":6.730000000000018,
"tempMin":5.830000000000041,
"tempMax":7.080000000000041}},

Periodic accelerometer

These accelerometer features are accelerometer features without gravity. They are inspired by the following article: https://towardsdatascience.com/feature-engineering-on-time-series-data-transforming-signal-data-of-a-smartphone-accelerometer-for-72cbe34b8a60 . This is a summary of all accelerometer data collected from sensorStartTime to sensorEndTime. In this window, multiple accelerometer data points are collected (amount equal to count). Using these data point, multiple summary statistics are computed.

{"sensorStartTime":1698491752180232,"sensorEndTime":1698491860180394,
"data":{"__type":"dk.cachet.carp.accelerationfeatures",
"count":64470,
"xMean":-0.0023555999429555564,
"yMean":-0.0016902002697385214,
"zMean":-0.01793158789239075,
"xStd":0.005979315421282742,
"yStd":0.005322634515174999,
"zStd":0.00463866964233508,
"xAad":0.00491037367800825,
"yAad":0.004275993257216244,
"zAad":0.003696482229852209,
"xMin":-0.02102006418630481,
"yMin":-0.020370287778787317,
"zMin":-0.03617789268493653,
"xMax":0.014925657492130996,
"yMax":0.016880098460242153,
"zMax":-0.0011396223306655885,
"xMaxMinDiff":0.03594572167843581,
"yMaxMinDiff":0.03725038623902947,
"zMaxMinDiff":0.03503827035427094,
"xMedian":-0.0019007288105785847,
"yMedian":-0.0017290993267670275,
"zMedian":-0.0179538756608963,
"xMad":0.004336532671004534,
"yMad":0.0036580729298293594,
"zMad":0.0031037020683288563,
"xIqr":0.008724018344655634,
"yIqr":0.007313816109672189,
"zIqr":0.006241902709007265,
"xNegCount":40260,
"yNegCount":40272,
"zNegCount":64470,
"xPosCount":24210,
"yPosCount":24198,
"zPosCount":0,
"xAboveMean":33972,
"yAboveMean":32070,
"zAboveMean":32160,
"xEnergy":0.000041301063998441844,
"yEnergy":0.00003118721513395636,
"zEnergy":0.0003430591003932556,
"avgResultAcceleration":0.0198636780610058,
"signalMagnitudeArea":0.027547146018205425}},

Stepcount (pedometer)

The amount of steps currently recorded. This may reset to 0 after phone restart.

{"sensorStartTime":1698493072171717,"data":{"__type":"dk.cachet.carp.stepcount","steps":18470}}

Activity

Information about the activity the participant is currently doing.

{"sensorStartTime":1698736494868871,
"data":{"__type":"dk.cachet.carp.activity",
"confidence":70,
"type":"WALKING"}}

Screen

On Android, provides an event when the participant turns his/her screen on or off.

{"sensorStartTime":1697452963984320,
"data":{
"__type":"dk.cachet.carp.screenevent",
"screenEvent":"SCREEN_OFF"}}

Battery

Data on battery/charging.

{"sensorStartTime":1697452965894334,
"data":{"__type":"dk.cachet.carp.batterystate",
"batteryLevel":55,
"batteryStatus":"discharging"}},

App usage

Data on the used apps in a specific time frame. For each app you get different features:

• package name (e.g. io.m_Path_Sense.kuleuven): the full name of the app. This would be the full name of the m-Path Sense app.
• usage: The amount an app is used in microseconds.
• lastForeground: the time the app was last in foreground (in local time for participant). When this is set to 1970, it means that the app was not recently in foreground or this foreground time was not recorded well by Android.
• startDate and endDate: app usage is always measured in a window. This startDate cannot be chosen, it is an internal variable of the phone. The endDate is approximately the current time for the participant.

{"sensorStartTime":1699270166384879,"sensorEndTime":1699270176237438,
"data":{"__type":"dk.cachet.carp.appusage","start":"2023-11-06T11:29:26.384879Z","end":"2023-11-06T11:29:36.237438Z","usage":
{"io.m_Path_Sense.kuleuven":{"packageName":"io.m_Path_Sense.kuleuven","appName":"kuleuven","usage":462000000,"startDate":"2023-11-06T00:20:13.000","endDate":"2023-11-06T12:26:25.000","lastForeground":"2023-11-06T12:26:25.000"},
"com.sec.android.app.launcher":{"packageName":"com.sec.android.app.launcher","appName":"launcher","usage":2105000000,"startDate":"2023-11-06T00:20:13.000","endDate":"2023-11-06T12:26:25.000","lastForeground":"1970-01-01T01:00:00.000"},
"com.samsung.android.app.spage":{"packageName":"com.samsung.android.app.spage","appName":"spage","usage":1000000,"startDate":"2023-11-06T00:20:13.000","endDate":"2023-11-06T12:29:33.000","lastForeground":"1970-01-01T01:00:00.000"},
"com.sec.android.app.clockpackage":{"packageName":"com.sec.android.app.clockpackage","appName":"clockpackage","usage":35000000,"startDate":"2023-11-06T00:20:13.000","endDate":"2023-11-06T12:26:25.000","lastForeground":"2023-11-06T07:35:23.000"},
"com.zhiliaoapp.musically":{"packageName":"com.zhiliaoapp.musically","appName":"musically","usage":593000000,"startDate":"2023-11-06T00:20:13.000","endDate":"2023-11-06T12:26:25.000","lastForeground":"1970-01-01T01:00:00.000"},
"com.samsung.android.app.galaxyfinder":{"packageName":"com.samsung.android.app.galaxyfinder","appName":"galaxyfinder","usage":1000000,"startDate":"2023-11-06T00:20:13.000","endDate":"2023-11-06T12:26:25.000","lastForeground":"1970-01-01T01:00:00.000"},
"com.google.android.permissioncontroller":{"packageName":"com.google.android.permissioncontroller","appName":"permissioncontroller","usage":9000000,"startDate":"2023-11-06T00:20:13.000","endDate":"2023-11-06T12:29:33.000","lastForeground":"1970-01-01T01:00:00.000"},"com.facebook.katana":{"packageName":"com.facebook.katana","appName":"katana","usage":40000000,"startDate":"2023-11-06T00:20:13.000","endDate":"2023-11-06T12:26:25.000","lastForeground":"1970-01-01T01:00:00.000"},
"com.samsung.android.calendar":{"packageName":"com.samsung.android.calendar","appName":"calendar","usage":125000000,"startDate":"2023-11-06T00:20:13.000","endDate":"2023-11-06T12:26:25.000","lastForeground":"1970-01-01T01:00:00.000"},"com.android.settings":{"packageName":"com.android.settings","appName":"settings","usage":8000000,"startDate":"2023-11-06T00:20:13.000","endDate":"2023-11-06T12:26:25.000","lastForeground":"1970-01-01T01:00:00.000"},
"com.samsung.android.messaging":{"packageName":"com.samsung.android.messaging","appName":"messaging","usage":8000000,"startDate":"2023-11-06T00:20:13.000","endDate":"2023-11-06T12:26:25.000","lastForeground":"1970-01-01T01:00:00.000"},
"com.sec.android.app.camera":{"packageName":"com.sec.android.app.camera","appName":"camera","usage":7000000,"startDate":"2023-11-06T00:20:13.000","endDate":"2023-11-06T12:26:25.000","lastForeground":"2023-11-06T08:06:11.000"},
"be.vrt.mobile.android.deredactie":{"packageName":"be.vrt.mobile.android.deredactie","appName":"deredactie","usage":128000000,"startDate":"2023-11-06T00:20:13.000","endDate":"2023-11-06T12:26:25.000","lastForeground":"1970-01-01T01:00:00.000"},
"com.google.android.youtube":{"packageName":"com.google.android.youtube","appName":"youtube","usage":414000000,"startDate":"2023-11-06T00:20:13.000","endDate":"2023-11-06T12:26:25.000","lastForeground":"1970-01-01T01:00:00.000"},
"be.kuleuven.icts.authenticator":{"packageName":"be.kuleuven.icts.authenticator","appName":"authenticator","usage":1007000000,"startDate":"2023-11-06T00:20:13.000","endDate":"2023-11-06T12:26:25.000","lastForeground":"1970-01-01T01:00:00.000"},"com.whatsapp":{"packageName":
"com.whatsapp","appName":"whatsapp","usage":143000000,"startDate":"2023-11-06T00:20:13.000","endDate":"2023-11-06T12:26:25.000","lastForeground":"2023-11-06T11:24:37.000"},"com.google.android.gm":{"packageName":"com.google.android.gm","appName":"gm","usage":72000000,"startDate":"2023-11-06T00:20:13.000","endDate":"2023-11-06T12:28:23.000","lastForeground":"1970-01-01T01:00:00.000"},
"com.coingecko.coingeckoapp":{"packageName":"com.coingecko.coingeckoapp","appName":"coingeckoapp","usage":47000000,"startDate":"2023-11-06T00:20:13.000","endDate":"2023-11-06T12:26:25.000","lastForeground":"1970-01-01T01:00:00.000"}}}},
]

Connectivity

The current state of connectivity

{"sensorStartTime":1698736622457390,
"data":{"__type":"dk.cachet.carp.connectivity",
"connectivityStatus":"wifi"}},

WIFI

Information about the current WIFI connection

{"sensorStartTime":1698736696501213,
"data":{"__type":"dk.cachet.carp.wifi",
"ip":"192.156.0.123"}}

Light

Information about the amount of light. From stat to end time, multiple lux measurements are done by the phone. These are summarized in several statistics.

{"sensorStartTime":1697452876526334,"sensorEndTime":1697452879529818,
"data":{"__type":"dk.cachet.carp.ambientlight",
"meanLux":1494.1866666666667,
"stdLux":61.59170795560787,
"minLux":1377,
"maxLux":1598}},

Memory

Information about the current free RAM memory on the phone in bytes.

{"sensorStartTime":1697453849620984,
"data":{"__type":"dk.cachet.carp.freememory",
"freePhysicalMemory":672206848,
"freeVirtualMemory":2838151168}}

Processing collected data with mpathsenser

m-Path Sense generates a significant amount of raw data in the form of JSON files, which may be zipped or unzipped. In order to process this data for research purposes, we have created the mpathsenser R package, which is available on CRAN. The mpathsenser package is designed to provide researchers with an efficient and convenient way to manage and manipulate the data generated by m-Path Sense.

With mpathsenser, researchers can import the raw data into an SQLite database, which allows for easy querying and manipulation of the data. The package also includes various functions for data analysis, such as the link() function, which enables researchers to link the timescales of the sensing data with the ESM data.

In order to assist researchers in utilizing mpathsenser, we have created a detailed “Get Started” vignette. This vignette provides step-by-step instructions on how to use the mpathsenser package and its various functions. By utilizing mpathsenser, researchers can streamline their data processing workflow and concentrate on their analysis with confidence.

What to expect

As for any sensing app, iOS and Android may kill or pause apps in an attempt to preserve the battery. This means that the actual sensing frequency only approximates the theoretical sensing frequency. Below we show two examples for the accelerometer on Android as well as iOS. The feature should be measured every 2 minutes, resulting in 30 measurements per hour. As shown in the following picture the actual amount of measurements per hour is more around 20.

Both phones were used as a phone would normally be used in daily life. When a notification was received noting that sensing may have stopped, the app was reopened as fast as possible. As shown in the picture, the sensing frequency specifically decreases during the night, when the phone is not used. On iOS, the sensing completely stopped for a few hours there. In general however, the data is sufficient for most purposes as when the participant is awake and using his/her phone, the coverage is relatively good.

Note that these are just examples of how the coverage can look like in a real experiment. Exact coverage depends on the phones, phone settings, study. For tips on how to optimize the use m-Path sense look at this section.