Node-RED Hosting in the Cloud with a Twitter Feed Analyzer Sample

    Share on FacebookShare on Google+Tweet about this on TwitterShare on LinkedInEmail this to someone

    node red deployment exampleDue to applications and components in global IoT (The Internet of Things) ecosystem getting more and more interrelated, it becomes essential to be capable of wiring them together to achieve automation. Advanced connectivity of systems allows to eliminate repetitive programming tasks and therefore – to free time for research and development.

    One of the tools to make different systems interact with each other is Node-RED  – a simple programming tool for binding online services, hardware devices, and APIs in an interactive and visually-appealing fashion. It provides a browser-based editor and a wide set of nodes to build and deploy various functional flows.

    Within this article, we’ll take a look at a solution we’ve implemented for a quick Node-RED delivery and will deploy a sample flow of Twitter streaming data analyzer to store and alert upon posts on any theme you are interested in.

    A Simple Way to Run Node-RED Instance Within a Cloud

    At Jelastic, the ready-to-go Node-RED development environment is provided as a single-click installation package. It is available within Jelastic JPS Collection on GitHub alongside with other preconfigured solutions and could be installed on any Platform of 4.9.5 Jelastic version and higher.  

    All the required resources and settings for running Node-RED on Jelastic are declared within the package manifest.jps file, where the main points to consider are:

    • nodered/node-red-docker Docker image with the latest tag is used as a solution base
    • created Node-RED instance is assigned 16 dynamic cloudlets, which enable it to scale vertically up to 2 GiB of RAM and 6.4 GHz of CPU
    • automatically attached endpoint allows to access Node-RED admin panel through a randomly allocated port at Shared Load Balancer (eliminating the necessity to purchase Public IP address)

    node red twitter sentiment analysisSo, let’s bring it to life – log in to your Jelastic Cloud account and proceed with the following steps.  

    1. Within the main dashboard page, click Import at the top and switch to the URL tab. Here, insert link to the corresponding manifest.jps file:

    https://github.com/jelastic-jps/node-red/blob/master/manifest.jps node red sentiment analysisClick Import to continue.

    2. In the opened installation dialog box:

    • type Environment name to be used as its internal hostname
    • optionally, specify environment Display Name (so-called alias, for being shown within dashboard and SSH terminal)
    • select an environment Region (if multiple ones are available)

    node red github projectClick Install and wait a bit to be shown a notification message on this process completion. node red install to cloudClick Open in browser to access Node-RED development environment and start creating your first project (so-called Flow). access Node-RED development environmentFurther, we’ll learn how to interact with this wizard through a hands-on example of Twitter content tracker – building such a useful and practical tool will also help you to get started with Node-RED as quick as possible.

    Deploying Twitter Feed Analyzer as Node-RED Flow

    So, the application we are going to build is intended to search through Twitter feed upon a specified criteria (keyword, hashtag, and/or username), conduct sentiment analysis of observed posts and process them accordingly. Depending on the message character, the taken action should be:

    • upon detecting negative content – to alert via email message
    • for positive and neutral results – to store a mention within a dedicated file inside a container, for each type of characteristic separately

    For analysis, we’ll use #Cloud, #CloudComputing, and #PaaS hashtags as a base for the corresponding hosting area investigation. Obviously, you are free to specify any other search parameters upon your requirements – e.g. your company name.

    The whole application structure will look like below:Node red application structureHere, the following component types (nodes) are involved:

    • Twitter used to authenticate with your Twitter account and specify search keywords
    • Sentiment – designed to analyze posts and return a corresponding value (negative, positive or neutral)
    • Switch – intended to evaluate sentiment scores and take a corresponding action according to the defined tone
    • two File Nodes – provided to store positive and negative messages
    • Email – allows receiving instant alert messages with negative feedback

    Altogether, such a flow enables you to quickly react to negative feedback and, simultaneously, to learn about user experience, best practices, news, and events within neutral and positive posts. This way, building such a tool will help you to get relevant information in time and stay tuned to what’s happening in the market.

    So, below we’ll consider the parameters that should be adjusted for each of these node types and consequently implement the described above workflow.

    Twitter Node

    Within the Node-RED flow editor, expand the social section and drag&drop the twitter icon to place it on the sheet. This will be the very base of our application.node red installer Double-click the picked node and specify the following information in the opened dialog box:

    • Twitter ID – click on the pencil icon to Authorise app with your Twitter account login and password; after this operation successfully passes, select Add to finish
    • Search – choose an area for conducting search – for example, all public tweets
    • for specify a search term (keyword, hashtag, and/or username); multiple entries сould be separated with spaces (for the bot to look for their joint mention) or commas without spaces (to detect separate terms)
    • Name – optionally, give some custom name to the node (or leave the default one)
      node red githubSelect Done above to proceed.
    Tip: For more comprehensive feedback analysis, append an additional debug node to the main twitter one so that fetched posts could be reviewed directly through the editor.node red twitterAs a result, records of all semantic types will be available within the sidebar debug tab, being stored within its default msg.payload log.

    Sentiment Node

    Now, expand the analysis section and choose a sentiment node – it will analyze found posts and return a corresponding sentiment value to determine whether a tweet is negative, positive or neutral.node red running on cloudThis component requires no additional adjustments unless you’d like to change its name (otherwise, double-click this block, specify the preferred denomination and click Done).

    Switch Node

    Now let’s add another switch element from the function section. It is intended to evaluate the received sentiment score and take a corresponding action according to the defined tone. node red cloud hosting

    Tip: The average sentiment score ranges from -5 (which means a very negative tweet) to +5 (a very positive feedback), with 0 being a neutral value. Scores outside this range are considered as extreme values.

    To adjust the switch node configs, double-click it and provide the following details in the opened dialog box:

    • Name – type custom node name (or leave the default one)
    • Property – specify the msg.sentiment.score value and +add the following rules to determine which sentimental value should be considered as neutral, positive or negative respectively:
      • == 0
      • > 0
      • < 0 

    node-red installSelect checking all rules at the box bottom and click Done.

    File Nodes

    Now, we are going to set up logging of the observed neutral and complimentary messages. But before the corresponding node insertion, you need to create file(s) for such mentions to be stored in – thus, open Config manager for your Node-RED container and enter the desired destination directory.
    node red development environment Here, select New file within options menu, type a name for it (e.g. neutral) and press Enter. Repeat these actions to create another file for positive tweets.

    Note: Your application should have sufficient permissions to access and write to the created files. To ensure this, connect to your Node-RED container via SSH Gate as follows:

    Once the connection is established, run the chown node-red {filename} command within the destination directory.
    node red deploying flows

    Then, expand storage wizard section and drag the comprised  file node to the sheet. expand storage node redDouble-click this block to adjust the appropriate parameters:

    • Filename – specify a full path to a destination storing file
    • Action – select append to file to add new tweets to the end of a file. Optionally, tick  the following additional options are available for this settings:
      • Add newline (\n) to each payload? – for a new line to be added to every message (can be turned off if required)
      • Create directory if it doesn’t exist? – for a new directory to be created if it doesn’t exist
    • Name – enter index number of the required switch node condition so that tweets with the corresponding semantic value will be written to this file (e.g. 1 for neutral posts in this example)

    install node redClick Done to confirm. Repeat these operations to configure one more file node that will process and save positive tweets.

    Email Node

    The last required email component could be located under the social section.email node redPlace it within the work area and double-click to provide the following details:

    • To – type an email address the alerts on negative tweets should be sent to
    • Server & Port – leave the default values (smtp.gmail.com host and 465 port number correspondingly)
    • Userid & Password – provide credentials to an email account from which the notifications will be sent
    • Name – type index number of the corresponding switch node condition according to the post type you’d like to receive the alerts on (in our case, semantic range for negative tweets was defined 3rd)

    Note: To ensure that the email node will definitely work properly, your email account (i.e. the one alerts will be sent from) should meet the following requirements:

    • have the Access to less secure apps setting turned ON
    • being protected with a strong 8-character password (containing uppercase/lowercase letters, numbers and symbols)

    edit email nodeWhen you are finished, click Done.

    Note: Due to inherent Twitter API limits, you could be shown the rate limit hit warning, appeared under the twitter node, if changing nodes settings and/or clicking Deploy too often. This does not affect the actual flow operability – just wait a minute for this warning to be replaced with your search keyword(s) instead.

    Checking Results

    So, your self-made Twitter bot has started to work, tracking the current posts and processing the information you’ve asked about. Subsequently, results of this analysis could be checked as follows:

    • for neutral/positive mentions – click Config for the Node-RED instance within your Jelastic dashboard and navigate to the appropriate file; its content can be reviewed just through the embedded file manager:
      node red docker container
    • for negative tweets – just track new incomings within your email box
      node red docker image

    That’s it! The Node-RED flow is now up and running to analyze streaming data from Twitter. Try it yourself by getting an account at one of Cloud Hosting Platforms.

    Have questions about the Node-RED development environment? Feel free to ask within the comments below or get in touch with our technical experts at Stackoverflow.
    Share on FacebookShare on Google+Tweet about this on TwitterShare on LinkedInEmail this to someone

    Leave a Reply

    Your email address will not be published.