Objectives

Upon completion of this lesson, you will be able to:

  • list the different methods of feature engineering
  • know when to apply different methods
  • consider what happens when data is not properly shaped

Introduction

Feature engineering is a key step in creating machine learning models. It is concerned with preparing the input dataset to ensure that it meets the requirements of the machine learning algorithm, improving the performance of machine learning models, and creating features from raw data that better represent the underlying problem to the predictive models.

In simpler terms, feature engineering is a process of transforming raw data into features that better represent the underlying problem to the predictive models, resulting in improved model accuracy on unseen data.

Features

Features in your data are important characteristics or attributes that, when processed and interpreted by a machine learning model, can help the model make accurate predictions. They can be anything from individual data points (like a customer’s age or income in a financial model) to calculated values based on raw data (like the length of a text string or the sum total of certain transactions).

In the context of machine learning, a feature is an individual, measurable property or characteristic of a phenomenon being observed. It’s essentially an input variable — the independent variable in the model. In simpler terms, features are the data points or attributes that we use to predict our target variable.

Alternative terms are: column, dimension, variable, attribute, property

For example, if you’re trying to predict the price of a house (the target variable), features could include the number of bedrooms, the size of the house in square feet, the age of the house, the location, and so on. Each of these features provides information that the model can use to learn patterns and, ultimately, predict house prices.

In the same vein, if you’re building a spam detection model, features could include the frequency of certain words, the email’s length, the time it was sent, whether it contains attachments, and more.

Feature Engineering

Feature engineering involves creating new features from existing data that can better represent the underlying problem to the predictive models, ultimately improving their performance. This might involve transforming existing features, creating new features through combinations of existing ones, or even using domain knowledge to create entirely new features.

Feature engineering requires domain knowledge of the specific field the model is being built for, as well as a solid understanding of the data. This is because creating effective new features involves an understanding of the relationship between existing features and the target variable we want to predict.

The process can involve a variety of tasks like:

  1. Variable Transformation: This includes tasks such as standardization (scaling your variable to have zero mean and unit variance), or normalization (scaling variables between a minimum and maximum value), and, when needed, transforming features to a normal (Gaussian) distribution.

  2. Variable / Feature Creation: This involves creating new, more complex features out of existing ones. This could be as simple as creating a “length of text” feature in a sentiment analysis model, or as complex as creating a “customer lifetime value” feature in a sales prediction model.

  3. Handling Missing Values: Not all features will have values for all data points. Deciding on how to handle missing data is a crucial step in feature engineering.

  4. Handling Categorical Data: Many machine learning models require inputs to be numerical. Converting categorical data into a suitable numerical form is another common feature engineering task.

  5. Dealing with Outliers: Outliers, or values that are significantly different from most of the other values, can distort the performance of a machine learning model. Feature engineering often involves identifying and dealing with outliers, such as removing them or treating them as missing values and imputing a new value.

  6. Feature Grooming: Some features may not be relevant or may not contribute to the training of a model, so they are removed. It may also be the case, that there are too many features and the machine learning algorithm cannot handle the quantity of features, so some must be removed or combined into new features.

By leveraging domain knowledge, the feature engineering process can create more meaningful features and significantly boost the predictive power of machine learning models.

Consequences of Poor Engineering

If feature engineering is omitted from the process of developing a machine learning model, it could lead to a number of negative consequences:

  1. Poor Model Performance: Without feature engineering, the model may not perform as well as it could. This is because the raw data may not adequately represent the underlying structures and patterns that the model needs to learn. By creating new features or transforming existing ones, we can expose these structures and patterns more clearly, making it easier for the model to learn from them.

  2. Overfitting or Underfitting: Overfitting occurs when a model learns the training data too well and performs poorly on unseen data, while underfitting happens when the model doesn’t learn enough from the training data and performs poorly even on it. Feature engineering can help in dealing with both of these issues. For example, creating more meaningful features can reduce the dimensionality of the data, which can help in avoiding overfitting.

  3. Inefficiency: Without feature engineering, we might need to use more complex models to achieve the same performance. For example, with the right features, a linear model might be sufficient to get good results. Without feature engineering, we might need to use a much more computationally intensive model, like a neural network, to get the same performance.

  4. Lack of Interpretability: Feature engineering often results in models that are more interpretable. For example, if we use domain knowledge to create features that have a clear meaning, it can be easier to understand why the model is making the predictions it is.

  5. Ignoring Domain Knowledge: One of the biggest strengths of feature engineering is the ability to incorporate domain knowledge into the model. If we omit feature engineering, we miss out on this opportunity.

Overall, feature engineering is a crucial part of the machine learning process, and omitting it can lead to worse performance, inefficiency, overfitting, underfitting, and models that are hard to interpret.

Feature Engineering and CRISP-DM

The CRISP-DM (Cross-Industry Standard Process for Data Mining) framework consists of six phases:

  1. Business Understanding
  2. Data Understanding
  3. Data Preparation
  4. Modeling
  5. Evaluation
  6. Deployment

Feature engineering primarily occurs during the Data Preparation phase of the CRISP-DM framework. This is after you’ve developed an understanding of the business problem and the data that you have to work with, but before you begin modeling.

During the Data Preparation phase, you’ll perform tasks such as data cleaning, dealing with missing values, transforming variables, and creating new features. These activities are crucial for ensuring that your data is in the right format and condition for modeling.

However, it’s important to note that feature engineering is an iterative process. While the majority of feature engineering work often happens during Data Preparation, you might go back and perform additional feature engineering after doing some modeling, especially if the model performance is not satisfactory or if you get new insights during the modeling or evaluation phase.

For instance, you might find that a certain feature isn’t providing as much predictive power as you thought it would, or that a different transformation of a feature might improve the model’s performance. In these cases, you would go back to the Data Preparation phase, make the necessary adjustments, and then proceed with the modeling process again.

For more information on CRISP-DM, consult Lesson 5.104 / CRISP-DM Process for Data Analytics and Data Mining.

Feature Engineering Tasks

Variable Transformation

This includes tasks such as standardization (scaling your variable to have zero mean and unit variance), or normalization (scaling variables between a minimum and maximum value), and, when needed, transforming features to a normal (Gaussian) distribution.

For more information consult Lesson TBD.

Variable / Feature Creation

This involves creating new, more complex features out of existing ones. This could be as simple as creating a “length of text” feature in a sentiment analysis model, or as complex as creating a “customer lifetime value” feature in a sales prediction model.

For more information consult Lesson 3.206 / Normalizing Numeric Features for Machine Learning Algorithms.

Handling Missing Values

Not all features will have values for all data points. Deciding on how to handle missing data is a crucial step in feature engineering.

For more information consult Lesson 3.204 / Managing Missing Values in Data.

Handling Categorical Data

Many machine learning models require inputs to be numerical. Converting categorical data into a suitable numerical form is another common feature engineering task.

For more information consult Lesson 3.207 / Encoding Categorical Features.

Dealing with Outliers

Outliers, or values that are significantly different from most of the other values, can distort the performance of a machine learning model. Feature engineering often involves identifying and dealing with outliers, such as removing them or treating them as missing values and imputing a new value.

For more information consult Lesson 3.203 / Detecting and Managing Outliers.

Feature Grooming

Some features may not be relevant or may not contribute to the training of a model, so they are removed. It may also be the case, that there are too many features and the machine learning algorithm cannot handle the quantity of features, so some must be removed or combined into new features.

Tutorials

Summary

Feature engineering is a critical step in the machine learning pipeline where raw data is transformed into a suitable format to improve the performance of machine learning models. It involves creating new features from existing ones, or transforming features, based on domain knowledge and insights gained from the data.

A “feature” in the context of machine learning refers to an individual measurable property or characteristic of the phenomenon being observed. Essentially, it’s an input variable or an independent variable in the model that we use to predict our target variable.

Neglecting feature engineering can lead to various negative consequences such as poor model performance, overfitting or underfitting of the model, inefficiency in model training, and difficulty in model interpretability. It may also prevent the model from fully leveraging the domain knowledge encapsulated in the engineered features.

In the CRISP-DM (Cross-Industry Standard Process for Data Mining) framework, feature engineering primarily happens during the Data Preparation phase. This phase is after the Business Understanding and Data Understanding stages but before the Modeling phase. However, feature engineering is an iterative process and adjustments may be made after initial modeling and evaluation based on further insights and the performance of the model.

In summary, feature engineering is a critical, iterative process in machine learning model development that involves transforming and creating features to better represent the underlying problem to the model, improving model performance and interpretability.

Files & Resources

All Files for Lesson 3.202

Errata

None collected yet. Let us know.

LS0tCnRpdGxlOiAiT3ZlcnZpZXcgb2YgRmVhdHVyZSBFbmdpbmVlcmluZyIKcGFyYW1zOgogIHR5cGU6IGxlc3NvbgogIGNhdGVnb3J5OiAzCiAgc3RhY2tzOiAwCiAgbnVtYmVyOiAyMDIKICB0aW1lOiAzMAogIGxldmVsOiBiZWdpbm5lcgogIHRhZ3M6IG1pc3NpbmcgZGF0YSxtYWNoaW5lIGxlYXJuaW5nLGRhdGEgY2xlYW5pbmcsaW1wdXRhdGlvbixmZWF0dXJlIGVuZ2luZWVyaW5nLG91dGxpZXJzCiAgZGVzY3JpcHRpb246ICJQcm92aWRlcyBhbiBvdmVydmlldyBvZiB0aGUgbWV0aG9kcyBhbmQgcHVycG9zZSBvZiBmZWF0dXJlCiAgICAgICAgICAgICAgICBlbmdpbmVlcmluZyByZXF1aXJlZCB0byBzaGFwZSBkYXRhIGZvciB0aGUgdHJhaW5pbmcgb2YgCiAgICAgICAgICAgICAgICBtYWNoaW5lIGxlYXJuaW5nIG1vZGVscy4iCmRhdGU6ICI8c21hbGw+YHIgU3lzLkRhdGUoKWA8L3NtYWxsPiIKYXV0aG9yOiAiPHNtYWxsPk1hcnRpbiBTY2hlZGxiYXVlcjwvc21hbGw+IgplbWFpbDogIm0uc2NoZWRsYmF1ZXJAbmV1LmVkdSIKYWZmaWxpdGF0aW9uOiAiTm9ydGhlYXN0ZXJuIFVuaXZlcnNpdHkiCm91dHB1dDogCiAgYm9va2Rvd246Omh0bWxfZG9jdW1lbnQyOgogICAgdG9jOiB0cnVlCiAgICB0b2NfZmxvYXQ6IHRydWUKICAgIGNvbGxhcHNlZDogZmFsc2UKICAgIG51bWJlcl9zZWN0aW9uczogZmFsc2UKICAgIGNvZGVfZG93bmxvYWQ6IHRydWUKICAgIHRoZW1lOiBzcGFjZWxhYgogICAgaGlnaGxpZ2h0OiB0YW5nbwotLS0KCi0tLQp0aXRsZTogIjxzbWFsbD5gciBwYXJhbXMkY2F0ZWdvcnlgLmByIHBhcmFtcyRudW1iZXJgPC9zbWFsbD48YnIvPjxzcGFuIHN0eWxlPSdjb2xvcjogIzJFNDA1MzsgZm9udC1zaXplOiAwLjllbSc+YHIgcm1hcmtkb3duOjptZXRhZGF0YSR0aXRsZWA8L3NwYW4+IgotLS0KCmBgYHtyIGNvZGU9eGZ1bjo6cmVhZF91dGY4KHBhc3RlMChoZXJlOjpoZXJlKCksJy9SL19pbnNlcnQyREIuUicpKSwgaW5jbHVkZSA9IEZBTFNFfQpgYGAKCi0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQoKIyMgT2JqZWN0aXZlcwoKVXBvbiBjb21wbGV0aW9uIG9mIHRoaXMgbGVzc29uLCB5b3Ugd2lsbCBiZSBhYmxlIHRvOgoKLSAgIGxpc3QgdGhlIGRpZmZlcmVudCBtZXRob2RzIG9mIGZlYXR1cmUgZW5naW5lZXJpbmcKLSAgIGtub3cgd2hlbiB0byBhcHBseSBkaWZmZXJlbnQgbWV0aG9kcwotICAgY29uc2lkZXIgd2hhdCBoYXBwZW5zIHdoZW4gZGF0YSBpcyBub3QgcHJvcGVybHkgc2hhcGVkCgotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KCiMjIEludHJvZHVjdGlvbgoKRmVhdHVyZSBlbmdpbmVlcmluZyBpcyBhIGtleSBzdGVwIGluIGNyZWF0aW5nIG1hY2hpbmUgbGVhcm5pbmcgbW9kZWxzLiBJdCBpcyBjb25jZXJuZWQgd2l0aCBwcmVwYXJpbmcgdGhlIGlucHV0IGRhdGFzZXQgdG8gZW5zdXJlIHRoYXQgaXQgbWVldHMgdGhlIHJlcXVpcmVtZW50cyBvZiB0aGUgbWFjaGluZSBsZWFybmluZyBhbGdvcml0aG0sIGltcHJvdmluZyB0aGUgcGVyZm9ybWFuY2Ugb2YgbWFjaGluZSBsZWFybmluZyBtb2RlbHMsIGFuZCBjcmVhdGluZyBmZWF0dXJlcyBmcm9tIHJhdyBkYXRhIHRoYXQgYmV0dGVyIHJlcHJlc2VudCB0aGUgdW5kZXJseWluZyBwcm9ibGVtIHRvIHRoZSBwcmVkaWN0aXZlIG1vZGVscy4KCkluIHNpbXBsZXIgdGVybXMsIGZlYXR1cmUgZW5naW5lZXJpbmcgaXMgYSBwcm9jZXNzIG9mIHRyYW5zZm9ybWluZyByYXcgZGF0YSBpbnRvIGZlYXR1cmVzIHRoYXQgYmV0dGVyIHJlcHJlc2VudCB0aGUgdW5kZXJseWluZyBwcm9ibGVtIHRvIHRoZSBwcmVkaWN0aXZlIG1vZGVscywgcmVzdWx0aW5nIGluIGltcHJvdmVkIG1vZGVsIGFjY3VyYWN5IG9uIHVuc2VlbiBkYXRhLgoKIyMgRmVhdHVyZXMKCkZlYXR1cmVzIGluIHlvdXIgZGF0YSBhcmUgaW1wb3J0YW50IGNoYXJhY3RlcmlzdGljcyBvciBhdHRyaWJ1dGVzIHRoYXQsIHdoZW4gcHJvY2Vzc2VkIGFuZCBpbnRlcnByZXRlZCBieSBhIG1hY2hpbmUgbGVhcm5pbmcgbW9kZWwsIGNhbiBoZWxwIHRoZSBtb2RlbCBtYWtlIGFjY3VyYXRlIHByZWRpY3Rpb25zLiBUaGV5IGNhbiBiZSBhbnl0aGluZyBmcm9tIGluZGl2aWR1YWwgZGF0YSBwb2ludHMgKGxpa2UgYSBjdXN0b21lcidzIGFnZSBvciBpbmNvbWUgaW4gYSBmaW5hbmNpYWwgbW9kZWwpIHRvIGNhbGN1bGF0ZWQgdmFsdWVzIGJhc2VkIG9uIHJhdyBkYXRhIChsaWtlIHRoZSBsZW5ndGggb2YgYSB0ZXh0IHN0cmluZyBvciB0aGUgc3VtIHRvdGFsIG9mIGNlcnRhaW4gdHJhbnNhY3Rpb25zKS4KCkluIHRoZSBjb250ZXh0IG9mIG1hY2hpbmUgbGVhcm5pbmcsIGEgZmVhdHVyZSBpcyBhbiBpbmRpdmlkdWFsLCBtZWFzdXJhYmxlIHByb3BlcnR5IG9yIGNoYXJhY3RlcmlzdGljIG9mIGEgcGhlbm9tZW5vbiBiZWluZyBvYnNlcnZlZC4gSXQncyBlc3NlbnRpYWxseSBhbiBpbnB1dCB2YXJpYWJsZSAtLS0gdGhlIGluZGVwZW5kZW50IHZhcmlhYmxlIGluIHRoZSBtb2RlbC4gSW4gc2ltcGxlciB0ZXJtcywgZmVhdHVyZXMgYXJlIHRoZSBkYXRhIHBvaW50cyBvciBhdHRyaWJ1dGVzIHRoYXQgd2UgdXNlIHRvIHByZWRpY3Qgb3VyIHRhcmdldCB2YXJpYWJsZS4KCkFsdGVybmF0aXZlIHRlcm1zIGFyZTogY29sdW1uLCBkaW1lbnNpb24sIHZhcmlhYmxlLCBhdHRyaWJ1dGUsIHByb3BlcnR5CgpGb3IgZXhhbXBsZSwgaWYgeW91J3JlIHRyeWluZyB0byBwcmVkaWN0IHRoZSBwcmljZSBvZiBhIGhvdXNlICh0aGUgdGFyZ2V0IHZhcmlhYmxlKSwgZmVhdHVyZXMgY291bGQgaW5jbHVkZSB0aGUgbnVtYmVyIG9mIGJlZHJvb21zLCB0aGUgc2l6ZSBvZiB0aGUgaG91c2UgaW4gc3F1YXJlIGZlZXQsIHRoZSBhZ2Ugb2YgdGhlIGhvdXNlLCB0aGUgbG9jYXRpb24sIGFuZCBzbyBvbi4gRWFjaCBvZiB0aGVzZSBmZWF0dXJlcyBwcm92aWRlcyBpbmZvcm1hdGlvbiB0aGF0IHRoZSBtb2RlbCBjYW4gdXNlIHRvIGxlYXJuIHBhdHRlcm5zIGFuZCwgdWx0aW1hdGVseSwgcHJlZGljdCBob3VzZSBwcmljZXMuCgpJbiB0aGUgc2FtZSB2ZWluLCBpZiB5b3UncmUgYnVpbGRpbmcgYSBzcGFtIGRldGVjdGlvbiBtb2RlbCwgZmVhdHVyZXMgY291bGQgaW5jbHVkZSB0aGUgZnJlcXVlbmN5IG9mIGNlcnRhaW4gd29yZHMsIHRoZSBlbWFpbCdzIGxlbmd0aCwgdGhlIHRpbWUgaXQgd2FzIHNlbnQsIHdoZXRoZXIgaXQgY29udGFpbnMgYXR0YWNobWVudHMsIGFuZCBtb3JlLgoKIyMgRmVhdHVyZSBFbmdpbmVlcmluZwoKRmVhdHVyZSBlbmdpbmVlcmluZyBpbnZvbHZlcyBjcmVhdGluZyBuZXcgZmVhdHVyZXMgZnJvbSBleGlzdGluZyBkYXRhIHRoYXQgY2FuIGJldHRlciByZXByZXNlbnQgdGhlIHVuZGVybHlpbmcgcHJvYmxlbSB0byB0aGUgcHJlZGljdGl2ZSBtb2RlbHMsIHVsdGltYXRlbHkgaW1wcm92aW5nIHRoZWlyIHBlcmZvcm1hbmNlLiBUaGlzIG1pZ2h0IGludm9sdmUgdHJhbnNmb3JtaW5nIGV4aXN0aW5nIGZlYXR1cmVzLCBjcmVhdGluZyBuZXcgZmVhdHVyZXMgdGhyb3VnaCBjb21iaW5hdGlvbnMgb2YgZXhpc3Rpbmcgb25lcywgb3IgZXZlbiB1c2luZyBkb21haW4ga25vd2xlZGdlIHRvIGNyZWF0ZSBlbnRpcmVseSBuZXcgZmVhdHVyZXMuCgpGZWF0dXJlIGVuZ2luZWVyaW5nIHJlcXVpcmVzIGRvbWFpbiBrbm93bGVkZ2Ugb2YgdGhlIHNwZWNpZmljIGZpZWxkIHRoZSBtb2RlbCBpcyBiZWluZyBidWlsdCBmb3IsIGFzIHdlbGwgYXMgYSBzb2xpZCB1bmRlcnN0YW5kaW5nIG9mIHRoZSBkYXRhLiBUaGlzIGlzIGJlY2F1c2UgY3JlYXRpbmcgZWZmZWN0aXZlIG5ldyBmZWF0dXJlcyBpbnZvbHZlcyBhbiB1bmRlcnN0YW5kaW5nIG9mIHRoZSByZWxhdGlvbnNoaXAgYmV0d2VlbiBleGlzdGluZyBmZWF0dXJlcyBhbmQgdGhlIHRhcmdldCB2YXJpYWJsZSB3ZSB3YW50IHRvIHByZWRpY3QuCgpUaGUgcHJvY2VzcyBjYW4gaW52b2x2ZSBhIHZhcmlldHkgb2YgdGFza3MgbGlrZToKCjEuICAqKlZhcmlhYmxlIFRyYW5zZm9ybWF0aW9uOioqIFRoaXMgaW5jbHVkZXMgdGFza3Mgc3VjaCBhcyBzdGFuZGFyZGl6YXRpb24gKHNjYWxpbmcgeW91ciB2YXJpYWJsZSB0byBoYXZlIHplcm8gbWVhbiBhbmQgdW5pdCB2YXJpYW5jZSksIG9yIG5vcm1hbGl6YXRpb24gKHNjYWxpbmcgdmFyaWFibGVzIGJldHdlZW4gYSBtaW5pbXVtIGFuZCBtYXhpbXVtIHZhbHVlKSwgYW5kLCB3aGVuIG5lZWRlZCwgdHJhbnNmb3JtaW5nIGZlYXR1cmVzIHRvIGEgbm9ybWFsIChHYXVzc2lhbikgZGlzdHJpYnV0aW9uLgoKMi4gICoqVmFyaWFibGUgLyBGZWF0dXJlIENyZWF0aW9uOioqIFRoaXMgaW52b2x2ZXMgY3JlYXRpbmcgbmV3LCBtb3JlIGNvbXBsZXggZmVhdHVyZXMgb3V0IG9mIGV4aXN0aW5nIG9uZXMuIFRoaXMgY291bGQgYmUgYXMgc2ltcGxlIGFzIGNyZWF0aW5nIGEgImxlbmd0aCBvZiB0ZXh0IiBmZWF0dXJlIGluIGEgc2VudGltZW50IGFuYWx5c2lzIG1vZGVsLCBvciBhcyBjb21wbGV4IGFzIGNyZWF0aW5nIGEgImN1c3RvbWVyIGxpZmV0aW1lIHZhbHVlIiBmZWF0dXJlIGluIGEgc2FsZXMgcHJlZGljdGlvbiBtb2RlbC4KCjMuICAqKkhhbmRsaW5nIE1pc3NpbmcgVmFsdWVzOioqIE5vdCBhbGwgZmVhdHVyZXMgd2lsbCBoYXZlIHZhbHVlcyBmb3IgYWxsIGRhdGEgcG9pbnRzLiBEZWNpZGluZyBvbiBob3cgdG8gaGFuZGxlIG1pc3NpbmcgZGF0YSBpcyBhIGNydWNpYWwgc3RlcCBpbiBmZWF0dXJlIGVuZ2luZWVyaW5nLgoKNC4gICoqSGFuZGxpbmcgQ2F0ZWdvcmljYWwgRGF0YToqKiBNYW55IG1hY2hpbmUgbGVhcm5pbmcgbW9kZWxzIHJlcXVpcmUgaW5wdXRzIHRvIGJlIG51bWVyaWNhbC4gQ29udmVydGluZyBjYXRlZ29yaWNhbCBkYXRhIGludG8gYSBzdWl0YWJsZSBudW1lcmljYWwgZm9ybSBpcyBhbm90aGVyIGNvbW1vbiBmZWF0dXJlIGVuZ2luZWVyaW5nIHRhc2suCgo1LiAgKipEZWFsaW5nIHdpdGggT3V0bGllcnM6KiogT3V0bGllcnMsIG9yIHZhbHVlcyB0aGF0IGFyZSBzaWduaWZpY2FudGx5IGRpZmZlcmVudCBmcm9tIG1vc3Qgb2YgdGhlIG90aGVyIHZhbHVlcywgY2FuIGRpc3RvcnQgdGhlIHBlcmZvcm1hbmNlIG9mIGEgbWFjaGluZSBsZWFybmluZyBtb2RlbC4gRmVhdHVyZSBlbmdpbmVlcmluZyBvZnRlbiBpbnZvbHZlcyBpZGVudGlmeWluZyBhbmQgZGVhbGluZyB3aXRoIG91dGxpZXJzLCBzdWNoIGFzIHJlbW92aW5nIHRoZW0gb3IgdHJlYXRpbmcgdGhlbSBhcyBtaXNzaW5nIHZhbHVlcyBhbmQgaW1wdXRpbmcgYSBuZXcgdmFsdWUuCgo2LiAgKipGZWF0dXJlIEdyb29taW5nKio6IFNvbWUgZmVhdHVyZXMgbWF5IG5vdCBiZSByZWxldmFudCBvciBtYXkgbm90IGNvbnRyaWJ1dGUgdG8gdGhlIHRyYWluaW5nIG9mIGEgbW9kZWwsIHNvIHRoZXkgYXJlIHJlbW92ZWQuIEl0IG1heSBhbHNvIGJlIHRoZSBjYXNlLCB0aGF0IHRoZXJlIGFyZSB0b28gbWFueSBmZWF0dXJlcyBhbmQgdGhlIG1hY2hpbmUgbGVhcm5pbmcgYWxnb3JpdGhtIGNhbm5vdCBoYW5kbGUgdGhlIHF1YW50aXR5IG9mIGZlYXR1cmVzLCBzbyBzb21lIG11c3QgYmUgcmVtb3ZlZCBvciBjb21iaW5lZCBpbnRvIG5ldyBmZWF0dXJlcy4KCkJ5IGxldmVyYWdpbmcgZG9tYWluIGtub3dsZWRnZSwgdGhlIGZlYXR1cmUgZW5naW5lZXJpbmcgcHJvY2VzcyBjYW4gY3JlYXRlIG1vcmUgbWVhbmluZ2Z1bCBmZWF0dXJlcyBhbmQgc2lnbmlmaWNhbnRseSBib29zdCB0aGUgcHJlZGljdGl2ZSBwb3dlciBvZiBtYWNoaW5lIGxlYXJuaW5nIG1vZGVscy4KCiMjIENvbnNlcXVlbmNlcyBvZiBQb29yIEVuZ2luZWVyaW5nCgpJZiBmZWF0dXJlIGVuZ2luZWVyaW5nIGlzIG9taXR0ZWQgZnJvbSB0aGUgcHJvY2VzcyBvZiBkZXZlbG9waW5nIGEgbWFjaGluZSBsZWFybmluZyBtb2RlbCwgaXQgY291bGQgbGVhZCB0byBhIG51bWJlciBvZiBuZWdhdGl2ZSBjb25zZXF1ZW5jZXM6CgoxLiAgKipQb29yIE1vZGVsIFBlcmZvcm1hbmNlOioqIFdpdGhvdXQgZmVhdHVyZSBlbmdpbmVlcmluZywgdGhlIG1vZGVsIG1heSBub3QgcGVyZm9ybSBhcyB3ZWxsIGFzIGl0IGNvdWxkLiBUaGlzIGlzIGJlY2F1c2UgdGhlIHJhdyBkYXRhIG1heSBub3QgYWRlcXVhdGVseSByZXByZXNlbnQgdGhlIHVuZGVybHlpbmcgc3RydWN0dXJlcyBhbmQgcGF0dGVybnMgdGhhdCB0aGUgbW9kZWwgbmVlZHMgdG8gbGVhcm4uIEJ5IGNyZWF0aW5nIG5ldyBmZWF0dXJlcyBvciB0cmFuc2Zvcm1pbmcgZXhpc3Rpbmcgb25lcywgd2UgY2FuIGV4cG9zZSB0aGVzZSBzdHJ1Y3R1cmVzIGFuZCBwYXR0ZXJucyBtb3JlIGNsZWFybHksIG1ha2luZyBpdCBlYXNpZXIgZm9yIHRoZSBtb2RlbCB0byBsZWFybiBmcm9tIHRoZW0uCgoyLiAgKipPdmVyZml0dGluZyBvciBVbmRlcmZpdHRpbmc6KiogT3ZlcmZpdHRpbmcgb2NjdXJzIHdoZW4gYSBtb2RlbCBsZWFybnMgdGhlIHRyYWluaW5nIGRhdGEgdG9vIHdlbGwgYW5kIHBlcmZvcm1zIHBvb3JseSBvbiB1bnNlZW4gZGF0YSwgd2hpbGUgdW5kZXJmaXR0aW5nIGhhcHBlbnMgd2hlbiB0aGUgbW9kZWwgZG9lc24ndCBsZWFybiBlbm91Z2ggZnJvbSB0aGUgdHJhaW5pbmcgZGF0YSBhbmQgcGVyZm9ybXMgcG9vcmx5IGV2ZW4gb24gaXQuIEZlYXR1cmUgZW5naW5lZXJpbmcgY2FuIGhlbHAgaW4gZGVhbGluZyB3aXRoIGJvdGggb2YgdGhlc2UgaXNzdWVzLiBGb3IgZXhhbXBsZSwgY3JlYXRpbmcgbW9yZSBtZWFuaW5nZnVsIGZlYXR1cmVzIGNhbiByZWR1Y2UgdGhlIGRpbWVuc2lvbmFsaXR5IG9mIHRoZSBkYXRhLCB3aGljaCBjYW4gaGVscCBpbiBhdm9pZGluZyBvdmVyZml0dGluZy4KCjMuICAqKkluZWZmaWNpZW5jeToqKiBXaXRob3V0IGZlYXR1cmUgZW5naW5lZXJpbmcsIHdlIG1pZ2h0IG5lZWQgdG8gdXNlIG1vcmUgY29tcGxleCBtb2RlbHMgdG8gYWNoaWV2ZSB0aGUgc2FtZSBwZXJmb3JtYW5jZS4gRm9yIGV4YW1wbGUsIHdpdGggdGhlIHJpZ2h0IGZlYXR1cmVzLCBhIGxpbmVhciBtb2RlbCBtaWdodCBiZSBzdWZmaWNpZW50IHRvIGdldCBnb29kIHJlc3VsdHMuIFdpdGhvdXQgZmVhdHVyZSBlbmdpbmVlcmluZywgd2UgbWlnaHQgbmVlZCB0byB1c2UgYSBtdWNoIG1vcmUgY29tcHV0YXRpb25hbGx5IGludGVuc2l2ZSBtb2RlbCwgbGlrZSBhIG5ldXJhbCBuZXR3b3JrLCB0byBnZXQgdGhlIHNhbWUgcGVyZm9ybWFuY2UuCgo0LiAgKipMYWNrIG9mIEludGVycHJldGFiaWxpdHk6KiogRmVhdHVyZSBlbmdpbmVlcmluZyBvZnRlbiByZXN1bHRzIGluIG1vZGVscyB0aGF0IGFyZSBtb3JlIGludGVycHJldGFibGUuIEZvciBleGFtcGxlLCBpZiB3ZSB1c2UgZG9tYWluIGtub3dsZWRnZSB0byBjcmVhdGUgZmVhdHVyZXMgdGhhdCBoYXZlIGEgY2xlYXIgbWVhbmluZywgaXQgY2FuIGJlIGVhc2llciB0byB1bmRlcnN0YW5kIHdoeSB0aGUgbW9kZWwgaXMgbWFraW5nIHRoZSBwcmVkaWN0aW9ucyBpdCBpcy4KCjUuICAqKklnbm9yaW5nIERvbWFpbiBLbm93bGVkZ2U6KiogT25lIG9mIHRoZSBiaWdnZXN0IHN0cmVuZ3RocyBvZiBmZWF0dXJlIGVuZ2luZWVyaW5nIGlzIHRoZSBhYmlsaXR5IHRvIGluY29ycG9yYXRlIGRvbWFpbiBrbm93bGVkZ2UgaW50byB0aGUgbW9kZWwuIElmIHdlIG9taXQgZmVhdHVyZSBlbmdpbmVlcmluZywgd2UgbWlzcyBvdXQgb24gdGhpcyBvcHBvcnR1bml0eS4KCk92ZXJhbGwsIGZlYXR1cmUgZW5naW5lZXJpbmcgaXMgYSBjcnVjaWFsIHBhcnQgb2YgdGhlIG1hY2hpbmUgbGVhcm5pbmcgcHJvY2VzcywgYW5kIG9taXR0aW5nIGl0IGNhbiBsZWFkIHRvIHdvcnNlIHBlcmZvcm1hbmNlLCBpbmVmZmljaWVuY3ksIG92ZXJmaXR0aW5nLCB1bmRlcmZpdHRpbmcsIGFuZCBtb2RlbHMgdGhhdCBhcmUgaGFyZCB0byBpbnRlcnByZXQuCgojIyBGZWF0dXJlIEVuZ2luZWVyaW5nIGFuZCBDUklTUC1ETQoKVGhlICoqQ1JJU1AtRE0qKiAoQ3Jvc3MtSW5kdXN0cnkgU3RhbmRhcmQgUHJvY2VzcyBmb3IgRGF0YSBNaW5pbmcpIGZyYW1ld29yayBjb25zaXN0cyBvZiBzaXggcGhhc2VzOgoKMS4gIEJ1c2luZXNzIFVuZGVyc3RhbmRpbmcKMi4gIERhdGEgVW5kZXJzdGFuZGluZwozLiAgRGF0YSBQcmVwYXJhdGlvbgo0LiAgTW9kZWxpbmcKNS4gIEV2YWx1YXRpb24KNi4gIERlcGxveW1lbnQKCkZlYXR1cmUgZW5naW5lZXJpbmcgcHJpbWFyaWx5IG9jY3VycyBkdXJpbmcgdGhlICoqRGF0YSBQcmVwYXJhdGlvbioqIHBoYXNlIG9mIHRoZSBDUklTUC1ETSBmcmFtZXdvcmsuIFRoaXMgaXMgYWZ0ZXIgeW91J3ZlIGRldmVsb3BlZCBhbiB1bmRlcnN0YW5kaW5nIG9mIHRoZSBidXNpbmVzcyBwcm9ibGVtIGFuZCB0aGUgZGF0YSB0aGF0IHlvdSBoYXZlIHRvIHdvcmsgd2l0aCwgYnV0IGJlZm9yZSB5b3UgYmVnaW4gbW9kZWxpbmcuCgpEdXJpbmcgdGhlIERhdGEgUHJlcGFyYXRpb24gcGhhc2UsIHlvdSdsbCBwZXJmb3JtIHRhc2tzIHN1Y2ggYXMgZGF0YSBjbGVhbmluZywgZGVhbGluZyB3aXRoIG1pc3NpbmcgdmFsdWVzLCB0cmFuc2Zvcm1pbmcgdmFyaWFibGVzLCBhbmQgY3JlYXRpbmcgbmV3IGZlYXR1cmVzLiBUaGVzZSBhY3Rpdml0aWVzIGFyZSBjcnVjaWFsIGZvciBlbnN1cmluZyB0aGF0IHlvdXIgZGF0YSBpcyBpbiB0aGUgcmlnaHQgZm9ybWF0IGFuZCBjb25kaXRpb24gZm9yIG1vZGVsaW5nLgoKSG93ZXZlciwgaXQncyBpbXBvcnRhbnQgdG8gbm90ZSB0aGF0IGZlYXR1cmUgZW5naW5lZXJpbmcgaXMgYW4gaXRlcmF0aXZlIHByb2Nlc3MuIFdoaWxlIHRoZSBtYWpvcml0eSBvZiBmZWF0dXJlIGVuZ2luZWVyaW5nIHdvcmsgb2Z0ZW4gaGFwcGVucyBkdXJpbmcgRGF0YSBQcmVwYXJhdGlvbiwgeW91IG1pZ2h0IGdvIGJhY2sgYW5kIHBlcmZvcm0gYWRkaXRpb25hbCBmZWF0dXJlIGVuZ2luZWVyaW5nIGFmdGVyIGRvaW5nIHNvbWUgbW9kZWxpbmcsIGVzcGVjaWFsbHkgaWYgdGhlIG1vZGVsIHBlcmZvcm1hbmNlIGlzIG5vdCBzYXRpc2ZhY3Rvcnkgb3IgaWYgeW91IGdldCBuZXcgaW5zaWdodHMgZHVyaW5nIHRoZSBtb2RlbGluZyBvciBldmFsdWF0aW9uIHBoYXNlLgoKRm9yIGluc3RhbmNlLCB5b3UgbWlnaHQgZmluZCB0aGF0IGEgY2VydGFpbiBmZWF0dXJlIGlzbid0IHByb3ZpZGluZyBhcyBtdWNoIHByZWRpY3RpdmUgcG93ZXIgYXMgeW91IHRob3VnaHQgaXQgd291bGQsIG9yIHRoYXQgYSBkaWZmZXJlbnQgdHJhbnNmb3JtYXRpb24gb2YgYSBmZWF0dXJlIG1pZ2h0IGltcHJvdmUgdGhlIG1vZGVsJ3MgcGVyZm9ybWFuY2UuIEluIHRoZXNlIGNhc2VzLCB5b3Ugd291bGQgZ28gYmFjayB0byB0aGUgRGF0YSBQcmVwYXJhdGlvbiBwaGFzZSwgbWFrZSB0aGUgbmVjZXNzYXJ5IGFkanVzdG1lbnRzLCBhbmQgdGhlbiBwcm9jZWVkIHdpdGggdGhlIG1vZGVsaW5nIHByb2Nlc3MgYWdhaW4uCgpGb3IgbW9yZSBpbmZvcm1hdGlvbiBvbiBDUklTUC1ETSwgY29uc3VsdCBbTGVzc29uIDUuMTA0IC8gQ1JJU1AtRE0gUHJvY2VzcyBmb3IgRGF0YSBBbmFseXRpY3MgYW5kIERhdGEgTWluaW5nXShodHRwOi8vYXJ0aWZpY2l1bS51cy9sZXNzb25zLzA1LmRtL2wtNS0xMDQtY3Jpc3AtZG0vbC01LTEwNC5odG1sKS4KCiMjIEZlYXR1cmUgRW5naW5lZXJpbmcgVGFza3MKCiMjIyBWYXJpYWJsZSBUcmFuc2Zvcm1hdGlvbgoKVGhpcyBpbmNsdWRlcyB0YXNrcyBzdWNoIGFzIHN0YW5kYXJkaXphdGlvbiAoc2NhbGluZyB5b3VyIHZhcmlhYmxlIHRvIGhhdmUgemVybyBtZWFuIGFuZCB1bml0IHZhcmlhbmNlKSwgb3Igbm9ybWFsaXphdGlvbiAoc2NhbGluZyB2YXJpYWJsZXMgYmV0d2VlbiBhIG1pbmltdW0gYW5kIG1heGltdW0gdmFsdWUpLCBhbmQsIHdoZW4gbmVlZGVkLCB0cmFuc2Zvcm1pbmcgZmVhdHVyZXMgdG8gYSBub3JtYWwgKEdhdXNzaWFuKSBkaXN0cmlidXRpb24uCgpGb3IgbW9yZSBpbmZvcm1hdGlvbiBjb25zdWx0IFtMZXNzb24gVEJEXSgpLgoKIyMjIFZhcmlhYmxlIC8gRmVhdHVyZSBDcmVhdGlvbgoKVGhpcyBpbnZvbHZlcyBjcmVhdGluZyBuZXcsIG1vcmUgY29tcGxleCBmZWF0dXJlcyBvdXQgb2YgZXhpc3Rpbmcgb25lcy4gVGhpcyBjb3VsZCBiZSBhcyBzaW1wbGUgYXMgY3JlYXRpbmcgYSAibGVuZ3RoIG9mIHRleHQiIGZlYXR1cmUgaW4gYSBzZW50aW1lbnQgYW5hbHlzaXMgbW9kZWwsIG9yIGFzIGNvbXBsZXggYXMgY3JlYXRpbmcgYSAiY3VzdG9tZXIgbGlmZXRpbWUgdmFsdWUiIGZlYXR1cmUgaW4gYSBzYWxlcyBwcmVkaWN0aW9uIG1vZGVsLgoKRm9yIG1vcmUgaW5mb3JtYXRpb24gY29uc3VsdCBbTGVzc29uIDMuMjA2IC8gTm9ybWFsaXppbmcgTnVtZXJpYyBGZWF0dXJlcyBmb3IgTWFjaGluZSBMZWFybmluZyBBbGdvcml0aG1zXShodHRwOi8vYXJ0aWZpY2l1bS51cy9sZXNzb25zLzAzLm1sL2wtMy0yMDYtZmVhdHVyZS1ub3JtYWxpemF0aW9uL2wtMy0yMDYuaHRtbCkuCgojIyMgSGFuZGxpbmcgTWlzc2luZyBWYWx1ZXMKCk5vdCBhbGwgZmVhdHVyZXMgd2lsbCBoYXZlIHZhbHVlcyBmb3IgYWxsIGRhdGEgcG9pbnRzLiBEZWNpZGluZyBvbiBob3cgdG8gaGFuZGxlIG1pc3NpbmcgZGF0YSBpcyBhIGNydWNpYWwgc3RlcCBpbiBmZWF0dXJlIGVuZ2luZWVyaW5nLgoKRm9yIG1vcmUgaW5mb3JtYXRpb24gY29uc3VsdCBbTGVzc29uIDMuMjA0IC8gTWFuYWdpbmcgTWlzc2luZyBWYWx1ZXMgaW4gRGF0YV0oaHR0cDovL2FydGlmaWNpdW0udXMvbGVzc29ucy8wMy5tbC9sLTMtMjA0LW1pc3NpbmctdmFsdWVzL2wtMy0yMDQuaHRtbCkuCgojIyMgSGFuZGxpbmcgQ2F0ZWdvcmljYWwgRGF0YQoKTWFueSBtYWNoaW5lIGxlYXJuaW5nIG1vZGVscyByZXF1aXJlIGlucHV0cyB0byBiZSBudW1lcmljYWwuIENvbnZlcnRpbmcgY2F0ZWdvcmljYWwgZGF0YSBpbnRvIGEgc3VpdGFibGUgbnVtZXJpY2FsIGZvcm0gaXMgYW5vdGhlciBjb21tb24gZmVhdHVyZSBlbmdpbmVlcmluZyB0YXNrLgoKRm9yIG1vcmUgaW5mb3JtYXRpb24gY29uc3VsdCBbTGVzc29uIDMuMjA3IC8gRW5jb2RpbmcgQ2F0ZWdvcmljYWwgRmVhdHVyZXNdKGh0dHA6Ly9hcnRpZmljaXVtLnVzL2xlc3NvbnMvMDMubWwvbC0zLTIwNy1jYXRlZ29yaWNhbC1lbmNvZGluZy9sLTMtMjA3Lmh0bWwpLgoKIyMjIERlYWxpbmcgd2l0aCBPdXRsaWVycwoKT3V0bGllcnMsIG9yIHZhbHVlcyB0aGF0IGFyZSBzaWduaWZpY2FudGx5IGRpZmZlcmVudCBmcm9tIG1vc3Qgb2YgdGhlIG90aGVyIHZhbHVlcywgY2FuIGRpc3RvcnQgdGhlIHBlcmZvcm1hbmNlIG9mIGEgbWFjaGluZSBsZWFybmluZyBtb2RlbC4gRmVhdHVyZSBlbmdpbmVlcmluZyBvZnRlbiBpbnZvbHZlcyBpZGVudGlmeWluZyBhbmQgZGVhbGluZyB3aXRoIG91dGxpZXJzLCBzdWNoIGFzIHJlbW92aW5nIHRoZW0gb3IgdHJlYXRpbmcgdGhlbSBhcyBtaXNzaW5nIHZhbHVlcyBhbmQgaW1wdXRpbmcgYSBuZXcgdmFsdWUuCgpGb3IgbW9yZSBpbmZvcm1hdGlvbiBjb25zdWx0IFtMZXNzb24gMy4yMDMgLyBEZXRlY3RpbmcgYW5kIE1hbmFnaW5nIE91dGxpZXJzXShodHRwOi8vYXJ0aWZpY2l1bS51cy9sZXNzb25zLzAzLm1sL2wtMy0yMDMtb3V0bGllcnMvbC0zLTIwMy5odG1sKS4KCiMjIEZlYXR1cmUgR3Jvb21pbmcKClNvbWUgZmVhdHVyZXMgbWF5IG5vdCBiZSByZWxldmFudCBvciBtYXkgbm90IGNvbnRyaWJ1dGUgdG8gdGhlIHRyYWluaW5nIG9mIGEgbW9kZWwsIHNvIHRoZXkgYXJlIHJlbW92ZWQuIEl0IG1heSBhbHNvIGJlIHRoZSBjYXNlLCB0aGF0IHRoZXJlIGFyZSB0b28gbWFueSBmZWF0dXJlcyBhbmQgdGhlIG1hY2hpbmUgbGVhcm5pbmcgYWxnb3JpdGhtIGNhbm5vdCBoYW5kbGUgdGhlIHF1YW50aXR5IG9mIGZlYXR1cmVzLCBzbyBzb21lIG11c3QgYmUgcmVtb3ZlZCBvciBjb21iaW5lZCBpbnRvIG5ldyBmZWF0dXJlcy4KCiMjIFR1dG9yaWFscwoKPGlmcmFtZSBzcmM9Imh0dHBzOi8vcGxheWVyLnZpbWVvLmNvbS92aWRlby84MzM2NTAwNjA/aD0wNmE4M2M0NjcwIiB3aWR0aD0iNjQwIiBoZWlnaHQ9IjU2NCIgYWxsb3dmdWxsc2NyZWVuPSJhbGxvd2Z1bGxzY3JlZW4iIGFsbG93PSJhdXRvcGxheTsgZnVsbHNjcmVlbiIgZGF0YS1leHRlcm5hbD0iMSI+Cgo8L2lmcmFtZT4KCjxpZnJhbWUgc3JjPSJodHRwczovL3BsYXllci52aW1lby5jb20vdmlkZW8vODMzNjUwMTcxP2g9NDYyZjcxYzA5NSIgd2lkdGg9IjY0MCIgaGVpZ2h0PSI1NjQiIGFsbG93ZnVsbHNjcmVlbj0iYWxsb3dmdWxsc2NyZWVuIiBhbGxvdz0iYXV0b3BsYXk7IGZ1bGxzY3JlZW4iIGRhdGEtZXh0ZXJuYWw9IjEiPgoKPC9pZnJhbWU+Cgo8aWZyYW1lIHNyYz0iaHR0cHM6Ly9wbGF5ZXIudmltZW8uY29tL3ZpZGVvLzgzMzY1MDIyMT9oPTU4ZGUxMDJjMzMiIHdpZHRoPSI2NDAiIGhlaWdodD0iNTY0IiBhbGxvd2Z1bGxzY3JlZW49ImFsbG93ZnVsbHNjcmVlbiIgYWxsb3c9ImF1dG9wbGF5OyBmdWxsc2NyZWVuIiBkYXRhLWV4dGVybmFsPSIxIj4KCjwvaWZyYW1lPgoKPGlmcmFtZSBzcmM9Imh0dHBzOi8vcGxheWVyLnZpbWVvLmNvbS92aWRlby84MzM2NTAyNjY/aD1mY2M0MzVkNTkzIiB3aWR0aD0iNjQwIiBoZWlnaHQ9IjU2NCIgYWxsb3dmdWxsc2NyZWVuPSJhbGxvd2Z1bGxzY3JlZW4iIGFsbG93PSJhdXRvcGxheTsgZnVsbHNjcmVlbiIgZGF0YS1leHRlcm5hbD0iMSI+Cgo8L2lmcmFtZT4KCjxpZnJhbWUgc3JjPSJodHRwczovL3BsYXllci52aW1lby5jb20vdmlkZW8vODMzNjUwMzAwP2g9NWQ4NWNmYTYxYyIgd2lkdGg9IjY0MCIgaGVpZ2h0PSI1NjQiIGFsbG93ZnVsbHNjcmVlbj0iYWxsb3dmdWxsc2NyZWVuIiBhbGxvdz0iYXV0b3BsYXk7IGZ1bGxzY3JlZW4iIGRhdGEtZXh0ZXJuYWw9IjEiPgoKPC9pZnJhbWU+CgojIyBTdW1tYXJ5CgpGZWF0dXJlIGVuZ2luZWVyaW5nIGlzIGEgY3JpdGljYWwgc3RlcCBpbiB0aGUgbWFjaGluZSBsZWFybmluZyBwaXBlbGluZSB3aGVyZSByYXcgZGF0YSBpcyB0cmFuc2Zvcm1lZCBpbnRvIGEgc3VpdGFibGUgZm9ybWF0IHRvIGltcHJvdmUgdGhlIHBlcmZvcm1hbmNlIG9mIG1hY2hpbmUgbGVhcm5pbmcgbW9kZWxzLiBJdCBpbnZvbHZlcyBjcmVhdGluZyBuZXcgZmVhdHVyZXMgZnJvbSBleGlzdGluZyBvbmVzLCBvciB0cmFuc2Zvcm1pbmcgZmVhdHVyZXMsIGJhc2VkIG9uIGRvbWFpbiBrbm93bGVkZ2UgYW5kIGluc2lnaHRzIGdhaW5lZCBmcm9tIHRoZSBkYXRhLgoKQSAiZmVhdHVyZSIgaW4gdGhlIGNvbnRleHQgb2YgbWFjaGluZSBsZWFybmluZyByZWZlcnMgdG8gYW4gaW5kaXZpZHVhbCBtZWFzdXJhYmxlIHByb3BlcnR5IG9yIGNoYXJhY3RlcmlzdGljIG9mIHRoZSBwaGVub21lbm9uIGJlaW5nIG9ic2VydmVkLiBFc3NlbnRpYWxseSwgaXQncyBhbiBpbnB1dCB2YXJpYWJsZSBvciBhbiBpbmRlcGVuZGVudCB2YXJpYWJsZSBpbiB0aGUgbW9kZWwgdGhhdCB3ZSB1c2UgdG8gcHJlZGljdCBvdXIgdGFyZ2V0IHZhcmlhYmxlLgoKTmVnbGVjdGluZyBmZWF0dXJlIGVuZ2luZWVyaW5nIGNhbiBsZWFkIHRvIHZhcmlvdXMgbmVnYXRpdmUgY29uc2VxdWVuY2VzIHN1Y2ggYXMgcG9vciBtb2RlbCBwZXJmb3JtYW5jZSwgb3ZlcmZpdHRpbmcgb3IgdW5kZXJmaXR0aW5nIG9mIHRoZSBtb2RlbCwgaW5lZmZpY2llbmN5IGluIG1vZGVsIHRyYWluaW5nLCBhbmQgZGlmZmljdWx0eSBpbiBtb2RlbCBpbnRlcnByZXRhYmlsaXR5LiBJdCBtYXkgYWxzbyBwcmV2ZW50IHRoZSBtb2RlbCBmcm9tIGZ1bGx5IGxldmVyYWdpbmcgdGhlIGRvbWFpbiBrbm93bGVkZ2UgZW5jYXBzdWxhdGVkIGluIHRoZSBlbmdpbmVlcmVkIGZlYXR1cmVzLgoKSW4gdGhlIENSSVNQLURNIChDcm9zcy1JbmR1c3RyeSBTdGFuZGFyZCBQcm9jZXNzIGZvciBEYXRhIE1pbmluZykgZnJhbWV3b3JrLCBmZWF0dXJlIGVuZ2luZWVyaW5nIHByaW1hcmlseSBoYXBwZW5zIGR1cmluZyB0aGUgRGF0YSBQcmVwYXJhdGlvbiBwaGFzZS4gVGhpcyBwaGFzZSBpcyBhZnRlciB0aGUgQnVzaW5lc3MgVW5kZXJzdGFuZGluZyBhbmQgRGF0YSBVbmRlcnN0YW5kaW5nIHN0YWdlcyBidXQgYmVmb3JlIHRoZSBNb2RlbGluZyBwaGFzZS4gSG93ZXZlciwgZmVhdHVyZSBlbmdpbmVlcmluZyBpcyBhbiBpdGVyYXRpdmUgcHJvY2VzcyBhbmQgYWRqdXN0bWVudHMgbWF5IGJlIG1hZGUgYWZ0ZXIgaW5pdGlhbCBtb2RlbGluZyBhbmQgZXZhbHVhdGlvbiBiYXNlZCBvbiBmdXJ0aGVyIGluc2lnaHRzIGFuZCB0aGUgcGVyZm9ybWFuY2Ugb2YgdGhlIG1vZGVsLgoKSW4gc3VtbWFyeSwgZmVhdHVyZSBlbmdpbmVlcmluZyBpcyBhIGNyaXRpY2FsLCBpdGVyYXRpdmUgcHJvY2VzcyBpbiBtYWNoaW5lIGxlYXJuaW5nIG1vZGVsIGRldmVsb3BtZW50IHRoYXQgaW52b2x2ZXMgdHJhbnNmb3JtaW5nIGFuZCBjcmVhdGluZyBmZWF0dXJlcyB0byBiZXR0ZXIgcmVwcmVzZW50IHRoZSB1bmRlcmx5aW5nIHByb2JsZW0gdG8gdGhlIG1vZGVsLCBpbXByb3ZpbmcgbW9kZWwgcGVyZm9ybWFuY2UgYW5kIGludGVycHJldGFiaWxpdHkuCgotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KCiMjIEZpbGVzICYgUmVzb3VyY2VzCgpgYGB7ciB6aXBGaWxlcywgZWNobz1GQUxTRX0KemlwTmFtZSA9IHNwcmludGYoIkxlc3NvbkZpbGVzLSVzLSVzLnppcCIsIAogICAgICAgICAgICAgICAgIHBhcmFtcyRjYXRlZ29yeSwKICAgICAgICAgICAgICAgICBwYXJhbXMkbnVtYmVyKQoKdGV4dEFMaW5rID0gcGFzdGUwKCJBbGwgRmlsZXMgZm9yIExlc3NvbiAiLCAKICAgICAgICAgICAgICAgcGFyYW1zJGNhdGVnb3J5LCIuIixwYXJhbXMkbnVtYmVyKQoKIyBkb3dubG9hZEZpbGVzTGluaygpIGlzIGluY2x1ZGVkIGZyb20gX2luc2VydDJEQi5SCmtuaXRyOjpyYXdfaHRtbChkb3dubG9hZEZpbGVzTGluaygiLiIsIHppcE5hbWUsIHRleHRBTGluaykpCmBgYAoKLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCgojIyBMZWFybiBNb3JlCgotICAgW0xlc3NvbiAzLjIwNiAvIE5vcm1hbGl6aW5nIE51bWVyaWMgRmVhdHVyZXMgZm9yIE1hY2hpbmUgTGVhcm5pbmcgQWxnb3JpdGhtc10oaHR0cDovL2FydGlmaWNpdW0udXMvbGVzc29ucy8wMy5tbC9sLTMtMjA2LWZlYXR1cmUtbm9ybWFsaXphdGlvbi9sLTMtMjA2Lmh0bWwpCgotICAgW0xlc3NvbiAzLjIwNCAvIE1hbmFnaW5nIE1pc3NpbmcgVmFsdWVzIGluIERhdGFdKGh0dHA6Ly9hcnRpZmljaXVtLnVzL2xlc3NvbnMvMDMubWwvbC0zLTIwNC1taXNzaW5nLXZhbHVlcy9sLTMtMjA0Lmh0bWwpCgotICAgW0xlc3NvbiAzLjIwNyAvIEVuY29kaW5nIENhdGVnb3JpY2FsIEZlYXR1cmVzXShodHRwOi8vYXJ0aWZpY2l1bS51cy9sZXNzb25zLzAzLm1sL2wtMy0yMDctY2F0ZWdvcmljYWwtZW5jb2RpbmcvbC0zLTIwNy5odG1sKQoKLSAgIFtMZXNzb24gMy4yMDMgLyBEZXRlY3RpbmcgYW5kIE1hbmFnaW5nIE91dGxpZXJzXShodHRwOi8vYXJ0aWZpY2l1bS51cy9sZXNzb25zLzAzLm1sL2wtMy0yMDMtb3V0bGllcnMvbC0zLTIwMy5odG1sKQoKLSAgIFtMZXNzb24gNTUuMjAyIC8gRGF0YSBRdWFsaXR5LCBPdXRsaWVycywgYW5kIE1pc3NpbmcgRGF0YV0oaHR0cDovL2FydGlmaWNpdW0udXMvbGVzc29ucy81NS5kYXRhLWFuYWx5dGljcy9sLTU1LTIwMi1taXNzaW5nLWRhdGEtb3V0bGllcnMvbC01NS0yMDIuaHRtbCkKCiMjIEVycmF0YQoKTm9uZSBjb2xsZWN0ZWQgeWV0LiBbTGV0IHVzIGtub3ddKGh0dHBzOi8vZm9ybS5qb3Rmb3JtLmNvbS8yMTIxODcwNzI3ODQxNTcpe3RhcmdldD0iX2JsYW5rIn0uCgpgYGB7ciBjb2RlPXhmdW46OnJlYWRfdXRmOChwYXN0ZTAoaGVyZTo6aGVyZSgpLCcvUi9fZGVwbG95S25pdC5SJykpLCBpbmNsdWRlID0gRkFMU0V9CmBgYAo=