To manage the information obtained from the input data, the package uses an structure, called Instance, which allows store the extracted properties of the different Pipes. Below are described all the fields comprising the Instance class.

• source: The input data without modifications.
• date: The date on which the source was generated or sent.
• data: The input data with modifications.
• properties: Contains a list of properties extracted from the data that is being processed.
• path: Identifier of the input data.
• isValid: Indicates if the Instance is valid or not.
• flowPipes: The list contains the Pipes that the Instance has passed through.
• banPipes: The list contains the Pipes that can not be executed from that moment.

bdpar is able to load data source from multiple sources. In fact, by default is fully compatible with Twitter (twtid), SMS (tsms) and YouTube (ytbid) data sources. However, new data loaders can be easily implemented by implementing (i) a new class implementing obtainSource and obtainDate abstract methods from the Instance class and (ii) a new subclass overriding the createInstance method of the InstanceFactory class.

Is important to take into account that the type of Instance used is deducted by default according to the file extension. However, this behaviour can be easily modelled according to user needs.

library(R6)
library(pipeR)
library(readr)

## Warning: package 'readr' was built under R version 3.6.2

ExtractorTytb <- R6Class(
  classname = "ExtractorTytb",
  inherit = Instance,
  public = list(
    initialize = function(path) {
      if (!"character" %in% class(path)) {
        stop("[ExtractorTytb][initialize][Error]
                Checking the type of the variable: path ",
                  class(path))
      }
      path %>>%
        super$initialize()
    },
    obtainDate = function() {
      "" %>>%
        super$setDate()
      return()
    },
    obtainSource = function() {
      super$getPath() %>>%
      read_file() %>>%
      super$setSource()
        
    super$getSource() %>>%
      super$setData()
    
      return()
    }
  )
)

library(R6)
library(tools)
library(bdpar)
InstanceFactoryCustom <- R6Class(
  "InstanceFactoryCustom",
  public = list(
    initialize = function() {
    },
    createInstance = function(path) {
      if (!"character" %in% class(path)) {
        stop("[InstanceFactoryCustom][createInstance][Error]
                Checking the type of the variable: path ",
                  class(path))
      }
      switch(file_ext(path),
       `email` =  return(ExtractorEml$new(path)),
       `tytb` = return(ExtractorTytb$new(path))
      )
        
      return()
    }
  )
)

(i) Pipes of basic functionality

File2Pipe

Obtains the source using the obtainSource method which implements the subclass of the superclass Instance. By default, the subclass implemented are ExtractorEml, ExtractorSms, ExtractorTwtid and ExtractorYtbid.

FindEmojiPipe

Creates a new emoji property where the emojis stored that are in the data attribute are stored. In addition, you can decide if you want to modify the data property of the Instance, removing the emojis that are found.

FindEmoticonPipe

Creates a new emoticon property where the emoticons stored that are in the data attribute are stored. In addition, you can decide if you want to modify the data property of the Instance, removing the emoticons that are found.

FindHashtagPipe

Creates a new hashtag property where the hashtags stored that are in the data attribute are stored. In addition, you can decide if you want to modify the data property of the Instance, removing the hashtags that are found.

FindUrlPipe

Creates a new URLs property where the URLs stored that are in the data attribute are stored. In addition, you can decide if you want to modify the data property of the Instance, removing the URLs that are found.

FindUserNamePipe

Creates a new userName property where the user names stored that are in the data attribute are stored. In addition, you can decide if you want to modify the data property of the Instance, removing the user names that are found.

GuessDatePipe

Obtains the date using the obtainDate method which implements the subclass of the superclass Instance. By default, the subclass implemented are ExtractorEml, ExtractorSms, ExtractorTwtid and ExtractorYtbid.

GuessLanguagePipe

Guesses the language by using language detector of library cld2. Creates the language property which indicates the idiom text. Optionally, it is possible to choose the language provided by Twitter.

MeasureLengthPipe

Creates the length property which indicates the length of the text. The property’s name is customize throught the class constructor.

StoreFileExtPipe

Creates the extension property which indicates file’s extension.

TargetAssigningPipe

Identifies the class of the Instance (target attribute), starting with the path of the file.

TeeCSVPipe

Generates a CSV from the properties of the Instance.

ToLowerCasePipe

Converts the data of an Instance to lower case.

(ii) Pipes that access external files

AbbreviationPipe

Creates a new property abbreviation, where the abbreviations that are in the data attribute are stored. In addition, you can decide if you want to modify the data property of the Instance, replacing the abbreviations found for its extended version. The abbreviations and their substitutions have to be stored in files of type json the file associated with the language of the text is chosen. The loading and handling of the information of these files will be done through the ResourceHandler class.

ContractionPipe

Creates a new property contraction, where the contractions that are in the data attribute are stored. In addition, you can decide if you want to modify the data property of the Instance, replacing the contractions found for its extended version. The contractions and their substitutions have to be stored in files of type json the file associated with the language of the text is chosen. The loading and handling of the information of these files will be done through the ResourceHandler class.

InterjectionPipe

Creates a new property interjection, where the interjections that are in the data attribute are stored. In addition, you can decide if you want to modify the data property of the Instance, removing interjections. The interjections will be stored in files of type json and the file associated with the language of the text is chosen. The loading and handling of the information of these files will be done through the ResourceHandler class.

SlangPipe

Creates a new property slang, where the slang words that are in the data attribute are stored. In addition, you can decide if you want to modify the data property of the Instance, replacing the contractions for its extended version. The words slang and their substitutions hava to be stored in files of type json and the file associated with the language of the text is chosen. The loading and handling of the information of these files will be done through the ResourceHandler class.

StopWordPipe

Creates a new property stopwords, where the empty words that are in the data attribute are stored. In addition, you can decide if you want to modify the data property of the Instance, removing the empty words. The empty words will be stored in files of type json and the file associated with the language of the text is chosen. The loading and handling of the information of these files will be done through the ResourceHandler class.