Essays /

1569947283 Essay

Essay preview

IoT 2014 1569947283

Configuration of Smart Environments Made Simple
Combining Visual Modeling with Semantic Metadata and Reasoning Simon Mayer∗ , Nadine Inhelder∗ , Ruben Verborgh† , Rik Van de Walle† , and Friedemann Mattern∗ ∗ Institute

for Pervasive Computing, ETH Zurich, Zurich, Switzerland
Lab, Ghent University - iMinds, Ghent, Belgium

† Multimedia

static links between individual services: once created, they cannot adapt to a dynamic context where devices might become unavailable or new services might appear at runtime.

Abstract—We present an approach that combines semantic
metadata and reasoning with a visual modeling tool to enable the goal-driven configuration of smart environments for end users. In contrast to process-driven systems where service mashups
are statically defined, this approach makes use of embedded
semantic API descriptions to dynamically create mashups that fulfill the user’s goal. The main advantage of the presented system is its high degree of flexibility, as service mashups can adapt to dynamic environments and are fault-tolerant with respect to individual services becoming unavailable. To support end users in expressing their goals, we integrated a visual programming tool with our system. This tool enables users to model the

desired state of their smart environment graphically and thus hides the technicalities of the underlying semantics and the reasoning. Possible applications of the presented system include the configuration of smart homes to increase individual well-being, and reconfigurations of smart environments, for instance in the industrial automation or healthcare domains.


In this paper, we explore a novel method to facilitate
the composition of heterogeneous services for end users. We
propose a goal-driven approach, meaning that we ask users
to state which properties their smart environment should have (e.g., regarding their personal comfort, such as setting the ambient temperature). Given this statement of a user’s goal, our system determines whether the goal can be reached given

the set of available services, and also infers which user actions (i.e., HTTP requests in this case) are necessary to reach it. The user can then execute these requests and thereby modify
his environment to reach the desired goal. Because service
mashups are created at runtime from user goals, this approach can handle highly dynamic service environments.
To find out whether the system can provide a mashup that
achieves the user’s goal, it requires a way of determining what functionality the individual services provide and how to use them. Proposals of tackling this challenge include middleware platforms that are tightly integrated with a user’s smart home (e.g., [3]) and API directories such as ProgrammableWeb1 .

Because these, however, do not support the functionality-based discovery of services, we instead suggest to combine semantic information about what functionality a service provides with a description of its REST program interface (i.e. of how it can be invoked by clients) and expose this data using metadata

documents that are linked to the service URLs (see sections II-A and II-B for details about what these descriptions look like). However, since we cannot assume that users are familiar with semantic languages such as RDF, we integrated our system with a visual programming language that allows users to graphically specify the desired state of their smart environment (see Section II-C).


For a long time, researchers in academia and industry alike
have been focusing on bringing to life the vision of smart
environments, and in particular the smart home by integrating ideas from pervasive computing and the Internet of Things
(IoT) with “classical” home automation systems. Despite this grand effort to put residents in control of a growing number of smart devices in their home and workplace environments
and despite the necessary technologies being readily available, smart homes have not yet been widely adopted, which seems
to be caused to a large extent by the poor manageability and inflexibility of current home automation solutions [1]–[3]. Indeed, in environments populated by many heterogeneous
smart things that can modify the users’ environment (e.g., smart thermostats) or provide relevant contextual information (e.g., motion sensors), it is difficult for users to find and utilize services that provide the functionality they require [4]. This is especially true when considering more complex user

requirements that involve multiple co-operating devices within so-called “physical mashups,” i.e. composite applications that involve sensing or actuation of the physical world [5]. Enabling owners of smart homes to create such mashups has been

identified as an important domain in the field of ubiquitous computing research [1], and the easy configuration of smart
environments is also gaining importance in the context of
industrial automation in “smart factories” [6].

A. Use Case: The Smart Environments Configurator
The concrete use case that we will use as an example to
illustrate the components of our system throughout the rest of this paper is an application that automatically modifies smart environments to match end-user preferences. Using a handheld or wearable device such as a tablet computer, smartphone,

or smartwatch, users specify the song or radio station to be played in their environment and their comfort temperature,
set ambient alarms in the environment, or adjust the lighting to match their preferred levels. This device then negotiates with the environment to adjust the specified parameters to the user’s comfort settings and can also provide feedback (see Figure 1). Our goal is to enable such applications to operate successfully in arbitrary environments, i.e. not only in the user’s

To facilitate the configuration of smart environments for end users, others have applied graphical programming abstractions for home automation (e.g., [7], [8]) that allow users to create mashups that are able to perform complex tasks by integrating functionality across multiple devices or services – an excellent overview of such systems that also considers semantically

assisted mashup creation is presented in [9]. These approaches, however, take a process-driven approach where users create



a Web application that allows the user to configure his smart environment. This interface is also responsible for executing the requests that are proposed by the reasoner.


In the following, we describe the integration of our service discovery and look-up component with services that embed
functional semantic metadata in the form of RESTdesc descriptions [11]. We also show how a semantic reasoner can use such metadata to infer which mashups can be created in a smart
environment and the actions that these can perform for a user (Section II-A). Furthermore, we discuss an important extension to the RESTdesc format that guarantees its usability in the
context of smart environments while preserving the logical
integrity of the reasoning (Section II-B). Finally, we present how we integrated our approach with ClickScript3 , a visual
programming tool that enables end users to easily describe the desired state of their smart environment (Section II-C).

Fig. 1. Feedback of the Smart Environments Configurator application after attempting to modify a smart environment to match the user’s preferences with respect to lighting, audio playback, and the ambient temperature (mock-up).

private home but also in an office environment, in hotel rooms, cars, and potentially also in public places. In an industrial context, machines or assembly lines could automatically adjust to support their current...

Read more


/doc/draft-ietf-core-resource-directory/ /html/rfc5988 /restdesc/states# /teamsubmission/n3/ /tr/http-in-rdf10/ /tr/xlink/ 001 019 023 024 072 096 1 10 101 107 11 119 12 122 126 127 13 131 14 14th 15 1505 1514 1569947283 16 163 165 17 18 19 196 199 2 20 2000 2003 2006 2007 2009 2010 2011 2012 2013 2014 21 2115 2124 217 218 22 227 228 23 234 238 249 256 263 276 280 289 3 32 33 337 34 341627 352 384 3rd 4 40 5 511 520 526 536 57 5th 6 61 65 66 7 709 736 754 76 8 80 801 808 9 907 916 918 96 abil abl abstract academia access accomplish accord account achiev acknowledg acm across action activ actual actuat ad ad-hoc adapt add addit adj adjust adopt advantag advent advertis agarw ahead aid aim akesson alarm algorithm alik allow along alreadi also altern although altogeth alway ambient among annot answer anteced anymor apart api appear appli applic approach appropri arbitrari architectur ask aspect assembl assist associ assum asthmat attempt attribut audio author autom automat avail averag avoid b backward bahl barri base basic basler becom behavior belgium believ benefit berner berners-le better biggest bit block bodi bourn brace bridg bring brittl brows brush c cach california call cannot capabl captur car case catalog categori caus celsius celsius-to-fahrenheit certain chain challeng chang chi choos circumv classic clear clickscript clickscript3 client clinic cloud co co-oper co-ordin coap combin comfort commerci communic communiti compar complementari complex compliant complic compon compos composit comput conceiv concept concern concret conduct conf confid confidenti configur conflict coninx connect consensus consensus-bas consequ consid consider consist constant constrain constraint construct consum contain context contextu contradict contrast control convers convert convtemp coppen core correct correspond cost could crabtre creat creation current custom cval d data dbpedia de decad decentr decid decis declar default defin degre delam delay demonstr deploy deriv describ describedbi descript design desir despit detail deter determin deursen develop devic diagnost differ differenti difficult direct directori directory2 discov discoveri discuss display dissert distinguish distrib dixon doctor document domain domest done download drag driven due dynam e e.g earlier easi easili ecu ed edit editor efer effort elay element eleven emant emb embed employ enabl encapsul end end-us endpoint engin ensur entir entiti entri environ equival es especi eth euler evalu even eventu evolv ex exampl excel except exclus execut exist expect experi expert explicit exploit explor expos express extend extens extent eye f facilit fact factori fahrenheit fals familiar fashion fast faster fault fault-toler favorit feasibl featur feedback field fig figur final find first first-ord fit fix flexibl fli floerkemei focus follow form format former formul formula forward found foundat fragil freedom friedemann friend fulfil function functionality-bas fundament furthermor futur fval g gabarr gain generat generic geonam get ghent give given global goal goal-driven grand grant graph graphic greim ground group grow guarante guid guinard haerinck hall handheld handl hansson harper hastemp hasunit hasvalu hateoa header healthcar heart help hem heterogen hide hierarchi high high-level higher higher-level highest histori hoc hoeck hold home homeblox homey honor host hotel howev http humbl hundr hypermedia hypertext i-b i.e icon idea identifi ieee ii ii-a ii-b ii-c iii illustr imind imped implement implic import imposs improv includ incompat incorpor increas incur ind inde indic individu induc industri infer inflex inform informat infrastructur inheld input insight inss instanc instanti instead institut instruct int integr intel interact interest interfac internet interoper interpret intervent introduc introduct intuit investig invoc invok involv iot irvin iscuss issu iv j jos k kalasapur kleanthous know knowledg koleva kumar l lab lack languag larg lastra latter lazili lead learn lee leeway left length less let level liechti life light lightweight like limit line link linkbas linkbases10 literatur local locat locatedin log logic long look look-up lot luyten m machin machine-interpret made mahajan main major majorityor make manag mani manual manufactur map mar mark marshal mart mashup match mattern may mayer mean mechan media medic mehandjiev mention metadata metaphor method methodnam middlewar might millisecond mitig mix mobil mock mock-up model modifi moment monitor moreov motion move movement ms much multi multi-us multimedia multipl multitud must mutual mytemp n n3 nadin name namoun narrow nation navig necessari necessit need negat negoti network network-bas never new newli newtemp next ninja8 notat notation3 note novel nsdi number nviron o object obtain offer offic oldtemp omit on-the-fli onclus one ontolog ontology7 open oper opportun option order ordin orient other otiv outag output overview owl owner oxygen p pair pantofaru paper parallel paramet parent pars part parti particular path percom perfect perform person perspect pervas ph.d pham physic pictur pipe place plan platform play playback poor popul posit possibl postcondit potenti power pp pragmat precondit precondition/postcondition-style predic predicate-object prefer prefix presenc present preserv previous principl privaci privat probabl problem proc process process-driven program programmableweb1 project promis properti propos protocol prototyp provid public publish put q qiao qos quantif question r radio rapid rather rdf rdf/xml reach readili real real-world realiz reason reasoner-bas reconfigur reduc regard regular relat relation4 relev remain remedi remot remov repositori repres represent reqbodi request requesturi requir research resid resourc resourceconstrain resp respect respons rest restdesc restrict result return revisit rfid rietzler right rik rise roadmap robson robust rodden roo room root rout ruben rule run runtim sabl samea sarma saroiu satisfact satisfi satur scalabl scenario schaub scheme scienc scope screen search section secur see seem select semant semicolon send sens sensor sent separ serv servic serviceori set shadbolt sheng shipman shirazi shortcom show shown signific similar simon simpl simpli simplifi sinc situat skeptic smart smartphon smartwatch so-cal softwar solut solv song soto sound sourc special specif specifi springer st stack stakehold stand standard start state statechang statement static station steiner step still stipul straightforward structur style subject subtl success suffici suggest summar superset support surround swiss switzerland syntax syst system szabo tabl tablet tackl tailor takayama take target task technic technolog temp temp23 tempc temperatur tempf term test textual thank therebi therefor thermostat thing third third-parti thirdparti three throughout thus tight time togeth toler tool top topic total toward tran transit translat transmit tri trifa tripl true trust two type u ubicomp ubiquit unabl unavail under unit univers unpack upnp uri url usabl usag use used9 user user-configur user-friend util v valid vall valu van vanderhulst variabl vasilako verborgh via view viewpoint vision visual vocabulari vocabulary6 vol w wajid walch wall want warrant way wealth wearabl web web-bas web-en weber well well-b whether whole whose wide wiedersheim wild within without work workplac workshop world wot would write x xu yahoo yap yet yield z zurich