By using CiteSpace software to create a knowledge map of authors,institutions and keywords,the literature on the spatio-temporal behavior of Chinese residents based on big data in the architectural planning discipline...By using CiteSpace software to create a knowledge map of authors,institutions and keywords,the literature on the spatio-temporal behavior of Chinese residents based on big data in the architectural planning discipline published in the China Academic Network Publishing Database(CNKI)was analyzed and discussed.It is found that there was a lack of communication and cooperation among research institutions and scholars;the research hotspots involved four main areas,including“application in tourism research”,“application in traffic travel research”,“application in work-housing relationship research”,and“application in personal family life research”.展开更多
In the era of big data, huge volumes of data are generated from online social networks, sensor networks, mobile devices, and organizations’ enterprise systems. This phenomenon provides organizations with unprecedente...In the era of big data, huge volumes of data are generated from online social networks, sensor networks, mobile devices, and organizations’ enterprise systems. This phenomenon provides organizations with unprecedented opportunities to tap into big data to mine valuable business intelligence. However, traditional business analytics methods may not be able to cope with the flood of big data. The main contribution of this paper is the illustration of the development of a novel big data stream analytics framework named BDSASA that leverages a probabilistic language model to analyze the consumer sentiments embedded in hundreds of millions of online consumer reviews. In particular, an inference model is embedded into the classical language modeling framework to enhance the prediction of consumer sentiments. The practical implication of our research work is that organizations can apply our big data stream analytics framework to analyze consumers’ product preferences, and hence develop more effective marketing and production strategies.展开更多
Big data applications in healthcare have provided a variety of solutions to reduce costs,errors,and waste.This work aims to develop a real-time system based on big medical data processing in the cloud for the predicti...Big data applications in healthcare have provided a variety of solutions to reduce costs,errors,and waste.This work aims to develop a real-time system based on big medical data processing in the cloud for the prediction of health issues.In the proposed scalable system,medical parameters are sent to Apache Spark to extract attributes from data and apply the proposed machine learning algorithm.In this way,healthcare risks can be predicted and sent as alerts and recommendations to users and healthcare providers.The proposed work also aims to provide an effective recommendation system by using streaming medical data,historical data on a user’s profile,and a knowledge database to make themost appropriate real-time recommendations and alerts based on the sensor’s measurements.This proposed scalable system works by tweeting the health status attributes of users.Their cloud profile receives the streaming healthcare data in real time by extracting the health attributes via a machine learning prediction algorithm to predict the users’health status.Subsequently,their status can be sent on demand to healthcare providers.Therefore,machine learning algorithms can be applied to stream health care data from wearables and provide users with insights into their health status.These algorithms can help healthcare providers and individuals focus on health risks and health status changes and consequently improve the quality of life.展开更多
At this current time,data stream classification plays a key role in big data analytics due to its enormous growth.Most of the existing classification methods used ensemble learning,which is trustworthy but these metho...At this current time,data stream classification plays a key role in big data analytics due to its enormous growth.Most of the existing classification methods used ensemble learning,which is trustworthy but these methods are not effective to face the issues of learning from imbalanced big data,it also supposes that all data are pre-classified.Another weakness of current methods is that it takes a long evaluation time when the target data stream contains a high number of features.The main objective of this research is to develop a new method for incremental learning based on the proposed ant lion fuzzy-generative adversarial network model.The proposed model is implemented in spark architecture.For each data stream,the class output is computed at slave nodes by training a generative adversarial network with the back propagation error based on fuzzy bound computation.This method overcomes the limitations of existing methods as it can classify data streams that are slightly or completely unlabeled data and providing high scalability and efficiency.The results show that the proposed model outperforms stateof-the-art performance in terms of accuracy(0.861)precision(0.9328)and minimal MSE(0.0416).展开更多
On the basis of the digital Weifang geospatial framework,Smart Weifang spatio-temporal information cloud platform(WFCP)integrated legal person information,population,place name and address data,macroeconomic data and ...On the basis of the digital Weifang geospatial framework,Smart Weifang spatio-temporal information cloud platform(WFCP)integrated legal person information,population,place name and address data,macroeconomic data and so on.And it also expanded the data contents,such as the indoor and outdoor data,the overground and underground data,panoramic data and real data.It also introduced the contents of historical geographical information in different periods and real-time location information,address information of sensing equipment,real-time perception and interpreting information.It has overcome the difficulties of real-time access of Internet of Things(IoT)perception,multi-node collaboration,64-bit support,cluster deployment and has the characteristics of spatio-temporal management,ondemand service,large data analysis and micro-service architecture.It built spatio-temporal information big data center and spatio-temporal information cloud platform,realized the convergence and management of the distributed big data,deeply applied for land,transportation,environmental protection,police and subdistrict five areas,by supporting the integrated application of multi-source information and supporting intelligent deep application.In the aspect of hardware environment construction,according to the top-level design and unified arrangement of Smart Weifang,the WFCP was migrated to Weifang cloud computing center,to achieve the on-demand computing resources and dynamic scheduling load-based computing resources,to support the generalizing load map application.展开更多
The expanding amounts of information created by Internet of Things(IoT)devices places a strain on cloud computing,which is often used for data analysis and storage.This paper investigates a different approach based on...The expanding amounts of information created by Internet of Things(IoT)devices places a strain on cloud computing,which is often used for data analysis and storage.This paper investigates a different approach based on edge cloud applications,which involves data filtering and processing before being delivered to a backup cloud environment.This Paper suggest designing and implementing a low cost,low power cluster of Single Board Computers(SBC)for this purpose,reducing the amount of data that must be transmitted elsewhere,using Big Data ideas and technology.An Apache Hadoop and Spark Cluster that was used to run a test application was containerized and deployed using a Raspberry Pi cluster and Docker.To obtain system data and analyze the setup’s performance a Prometheusbased stack monitoring and alerting solution in the cloud based market is employed.This Paper assesses the system’s complexity and demonstrates how containerization can improve fault tolerance and maintenance ease,allowing the suggested solution to be used in industry.An evaluation of the overall performance is presented to highlight the capabilities and limitations of the suggested architecture,taking into consideration the suggested solution’s resource use in respect to device restrictions.展开更多
Handling sentiment drifts in real time twitter data streams are a challen-ging task while performing sentiment classifications,because of the changes that occur in the sentiments of twitter users,with respect to time....Handling sentiment drifts in real time twitter data streams are a challen-ging task while performing sentiment classifications,because of the changes that occur in the sentiments of twitter users,with respect to time.The growing volume of tweets with sentiment drifts has led to the need for devising an adaptive approach to detect and handle this drift in real time.This work proposes an adap-tive learning algorithm-based framework,Twitter Sentiment Drift Analysis-Bidir-ectional Encoder Representations from Transformers(TSDA-BERT),which introduces a sentiment drift measure to detect drifts and a domain impact score to adaptively retrain the classification model with domain relevant data in real time.The framework also works on static data by converting them to data streams using the Kafka tool.The experiments conducted on real time and simulated tweets of sports,health care andfinancial topics show that the proposed system is able to detect sentiment drifts and maintain the performance of the classification model,with accuracies of 91%,87%and 90%,respectively.Though the results have been provided only for a few topics,as a proof of concept,this framework can be applied to detect sentiment drifts and perform sentiment classification on real time data streams of any topic.展开更多
Marine big data are characterized by a large amount and complex structures,which bring great challenges to data management and retrieval.Based on the GeoSOT Grid Code and the composite index structure of the MongoDB d...Marine big data are characterized by a large amount and complex structures,which bring great challenges to data management and retrieval.Based on the GeoSOT Grid Code and the composite index structure of the MongoDB database,this paper proposes a spatio-temporal grid index model(STGI)for efficient optimized query of marine big data.A spatio-temporal secondary index is created on the spatial code and time code columns to build a composite index in the MongoDB database used for the storage of massive marine data.Multiple comparative experiments demonstrate that the retrieval efficiency adopting the STGI approach is increased by more than two to three times compared with other index models.Through theoretical analysis and experimental verification,the conclusion could be achieved that the STGI model is quite suitable for retrieving large-scale spatial data with low time frequency,such as marine big data.展开更多
With tremendous growing interests in Big Data, the performance improvement of Big Data systems becomes more and more important. Among many steps, the first one is to analyze and diagnose performance bottlenecks of the...With tremendous growing interests in Big Data, the performance improvement of Big Data systems becomes more and more important. Among many steps, the first one is to analyze and diagnose performance bottlenecks of the Big Data systems. Currently, there are two major solutions. One is the pure data-driven diagnosis approach, which may be very time-consuming;the other is the rule-based analysis method, which usually requires prior knowledge. For Big Data applications like Spark workloads, we observe that the tasks in the same stages normally execute the same or similar codes on each data partition. On basis of the stage similarity and distributed characteristics of Big Data systems, we analyze the behaviors of the Big Data applications in terms of both system and micro-architectural metrics of each stage. Furthermore, for different performance problems, we propose a hybrid approach that combines prior rules and machine learning algorithms to detect performance anomalies, such as straggler tasks, task assignment imbalance, data skew, abnormal nodes and outlier metrics. Following this methodology, we design and implement a lightweight, extensible tool, named HybridTune, and measure the overhead and anomaly detection effectiveness of HybridTune using the BigDataBench benchmarks. Our experiments show that the overhead of HybridTune is only 5%, and the accuracy of outlier detection algorithm reaches up to 93%. Finally, we report several use cases diagnosing Spark and Hadoop workloads using BigDataBench, which demonstrates the potential use of HybridTune.展开更多
文摘By using CiteSpace software to create a knowledge map of authors,institutions and keywords,the literature on the spatio-temporal behavior of Chinese residents based on big data in the architectural planning discipline published in the China Academic Network Publishing Database(CNKI)was analyzed and discussed.It is found that there was a lack of communication and cooperation among research institutions and scholars;the research hotspots involved four main areas,including“application in tourism research”,“application in traffic travel research”,“application in work-housing relationship research”,and“application in personal family life research”.
文摘In the era of big data, huge volumes of data are generated from online social networks, sensor networks, mobile devices, and organizations’ enterprise systems. This phenomenon provides organizations with unprecedented opportunities to tap into big data to mine valuable business intelligence. However, traditional business analytics methods may not be able to cope with the flood of big data. The main contribution of this paper is the illustration of the development of a novel big data stream analytics framework named BDSASA that leverages a probabilistic language model to analyze the consumer sentiments embedded in hundreds of millions of online consumer reviews. In particular, an inference model is embedded into the classical language modeling framework to enhance the prediction of consumer sentiments. The practical implication of our research work is that organizations can apply our big data stream analytics framework to analyze consumers’ product preferences, and hence develop more effective marketing and production strategies.
基金This study was financially supported by the Korea Health Technology R&D Project through the Korea Health Industry Development Institute(KHIDI),the Ministry of Health and Welfare(HI18C1216),and the Soonchunhyang University Research Fund.
文摘Big data applications in healthcare have provided a variety of solutions to reduce costs,errors,and waste.This work aims to develop a real-time system based on big medical data processing in the cloud for the prediction of health issues.In the proposed scalable system,medical parameters are sent to Apache Spark to extract attributes from data and apply the proposed machine learning algorithm.In this way,healthcare risks can be predicted and sent as alerts and recommendations to users and healthcare providers.The proposed work also aims to provide an effective recommendation system by using streaming medical data,historical data on a user’s profile,and a knowledge database to make themost appropriate real-time recommendations and alerts based on the sensor’s measurements.This proposed scalable system works by tweeting the health status attributes of users.Their cloud profile receives the streaming healthcare data in real time by extracting the health attributes via a machine learning prediction algorithm to predict the users’health status.Subsequently,their status can be sent on demand to healthcare providers.Therefore,machine learning algorithms can be applied to stream health care data from wearables and provide users with insights into their health status.These algorithms can help healthcare providers and individuals focus on health risks and health status changes and consequently improve the quality of life.
基金Taif University Researchers Supporting Project Number(TURSP-2020/126),Taif University,Taif,Saudi Arabia.
文摘At this current time,data stream classification plays a key role in big data analytics due to its enormous growth.Most of the existing classification methods used ensemble learning,which is trustworthy but these methods are not effective to face the issues of learning from imbalanced big data,it also supposes that all data are pre-classified.Another weakness of current methods is that it takes a long evaluation time when the target data stream contains a high number of features.The main objective of this research is to develop a new method for incremental learning based on the proposed ant lion fuzzy-generative adversarial network model.The proposed model is implemented in spark architecture.For each data stream,the class output is computed at slave nodes by training a generative adversarial network with the back propagation error based on fuzzy bound computation.This method overcomes the limitations of existing methods as it can classify data streams that are slightly or completely unlabeled data and providing high scalability and efficiency.The results show that the proposed model outperforms stateof-the-art performance in terms of accuracy(0.861)precision(0.9328)and minimal MSE(0.0416).
文摘On the basis of the digital Weifang geospatial framework,Smart Weifang spatio-temporal information cloud platform(WFCP)integrated legal person information,population,place name and address data,macroeconomic data and so on.And it also expanded the data contents,such as the indoor and outdoor data,the overground and underground data,panoramic data and real data.It also introduced the contents of historical geographical information in different periods and real-time location information,address information of sensing equipment,real-time perception and interpreting information.It has overcome the difficulties of real-time access of Internet of Things(IoT)perception,multi-node collaboration,64-bit support,cluster deployment and has the characteristics of spatio-temporal management,ondemand service,large data analysis and micro-service architecture.It built spatio-temporal information big data center and spatio-temporal information cloud platform,realized the convergence and management of the distributed big data,deeply applied for land,transportation,environmental protection,police and subdistrict five areas,by supporting the integrated application of multi-source information and supporting intelligent deep application.In the aspect of hardware environment construction,according to the top-level design and unified arrangement of Smart Weifang,the WFCP was migrated to Weifang cloud computing center,to achieve the on-demand computing resources and dynamic scheduling load-based computing resources,to support the generalizing load map application.
基金This research project was supported by a grant from the“Research Center of College of Computer and Information Sciences”,Deanship of Scientific Research,King Saud University.
文摘The expanding amounts of information created by Internet of Things(IoT)devices places a strain on cloud computing,which is often used for data analysis and storage.This paper investigates a different approach based on edge cloud applications,which involves data filtering and processing before being delivered to a backup cloud environment.This Paper suggest designing and implementing a low cost,low power cluster of Single Board Computers(SBC)for this purpose,reducing the amount of data that must be transmitted elsewhere,using Big Data ideas and technology.An Apache Hadoop and Spark Cluster that was used to run a test application was containerized and deployed using a Raspberry Pi cluster and Docker.To obtain system data and analyze the setup’s performance a Prometheusbased stack monitoring and alerting solution in the cloud based market is employed.This Paper assesses the system’s complexity and demonstrates how containerization can improve fault tolerance and maintenance ease,allowing the suggested solution to be used in industry.An evaluation of the overall performance is presented to highlight the capabilities and limitations of the suggested architecture,taking into consideration the suggested solution’s resource use in respect to device restrictions.
文摘Handling sentiment drifts in real time twitter data streams are a challen-ging task while performing sentiment classifications,because of the changes that occur in the sentiments of twitter users,with respect to time.The growing volume of tweets with sentiment drifts has led to the need for devising an adaptive approach to detect and handle this drift in real time.This work proposes an adap-tive learning algorithm-based framework,Twitter Sentiment Drift Analysis-Bidir-ectional Encoder Representations from Transformers(TSDA-BERT),which introduces a sentiment drift measure to detect drifts and a domain impact score to adaptively retrain the classification model with domain relevant data in real time.The framework also works on static data by converting them to data streams using the Kafka tool.The experiments conducted on real time and simulated tweets of sports,health care andfinancial topics show that the proposed system is able to detect sentiment drifts and maintain the performance of the classification model,with accuracies of 91%,87%and 90%,respectively.Though the results have been provided only for a few topics,as a proof of concept,this framework can be applied to detect sentiment drifts and perform sentiment classification on real time data streams of any topic.
基金This research was funded by the National Key Research and Development Plan(2018YFB0505300)the Guangxi Science and Technology Major Project(AA18118025)+1 种基金the Opening Foundation of Key Laboratory of Environment Change and Resources Use in Beibu Gulf,Ministry of Education(Nanning Normal University)Guangxi Key Laboratory of Earth Surface Processes and Intelligent Simulation(Nanning Normal University)(No.NNNU-KLOP-K1905).
文摘Marine big data are characterized by a large amount and complex structures,which bring great challenges to data management and retrieval.Based on the GeoSOT Grid Code and the composite index structure of the MongoDB database,this paper proposes a spatio-temporal grid index model(STGI)for efficient optimized query of marine big data.A spatio-temporal secondary index is created on the spatial code and time code columns to build a composite index in the MongoDB database used for the storage of massive marine data.Multiple comparative experiments demonstrate that the retrieval efficiency adopting the STGI approach is increased by more than two to three times compared with other index models.Through theoretical analysis and experimental verification,the conclusion could be achieved that the STGI model is quite suitable for retrieving large-scale spatial data with low time frequency,such as marine big data.
基金supported by the National Key Research and Development Program of China under Grant No.2016YFB1000601
文摘With tremendous growing interests in Big Data, the performance improvement of Big Data systems becomes more and more important. Among many steps, the first one is to analyze and diagnose performance bottlenecks of the Big Data systems. Currently, there are two major solutions. One is the pure data-driven diagnosis approach, which may be very time-consuming;the other is the rule-based analysis method, which usually requires prior knowledge. For Big Data applications like Spark workloads, we observe that the tasks in the same stages normally execute the same or similar codes on each data partition. On basis of the stage similarity and distributed characteristics of Big Data systems, we analyze the behaviors of the Big Data applications in terms of both system and micro-architectural metrics of each stage. Furthermore, for different performance problems, we propose a hybrid approach that combines prior rules and machine learning algorithms to detect performance anomalies, such as straggler tasks, task assignment imbalance, data skew, abnormal nodes and outlier metrics. Following this methodology, we design and implement a lightweight, extensible tool, named HybridTune, and measure the overhead and anomaly detection effectiveness of HybridTune using the BigDataBench benchmarks. Our experiments show that the overhead of HybridTune is only 5%, and the accuracy of outlier detection algorithm reaches up to 93%. Finally, we report several use cases diagnosing Spark and Hadoop workloads using BigDataBench, which demonstrates the potential use of HybridTune.
