Application Architect Changed Data-Architect Questions

The best solution to extract 50 million records from a custom object everyday from Salesforce org without facing query timeout issues is to use an ETL tool for extraction of records. ETL stands for extract, transform, and load, and it refers to a process of moving data from one system to another. An ETL tool is a software application that can connect to various data sources, perform data transformations, and load data into a target destination. ETL tools can handle large volumes of data efficiently and reliably, and they often provide features such as scheduling, monitoring, error handling, and logging5. Using a custom auto number and formula field and use that to chunk records while extracting data is a possible workaround, but it requires creating additional fields and writing complex queries. The REST API can extract data as it automatically chunks records by 200, but it has some limitations, such as a maximum of 50 million records per query job6. Asking SF support to increase the query timeout value is not feasible because query timeout values are not configurable

Setting up a Validation Rule to prevent blank values and marking the field required in setup at the field level are two approaches that the Architect should take to ensure that the field always contains data coming from the source system. A Validation Rule can display an error message when a record is created or edited with a blank value for the field, and prevent it from being saved. Marking the field required in setup at the field level can enforce the requirement for all records, regardless of the page layout or the source system. The other options are not effective or recommended for ensuring data quality, as they would either not prevent blank values, not apply to all records, or not address the root cause of the problem

Shield platform encryption is the recommended solution for encrypting sensitive patient records stored in files, as it provides encryption at rest for files and attachments, as well as standard and custom fields. Classic encryption only supports text fields with a maximum length of 175 characters, and does not encrypt files. Third-party App Exchange apps may not provide the same level of security and compliance as shield platform encryption. Storing files outside of salesforce may introduce additional complexity and latency

 Using hard delete in Bulk API, identifying records that have not been modified or used in last 3 years, and archiving the records in EDW before deleting from Salesforce are all good recommendations to reduce the number of records from the org1. Using hard delete in batch Apex or Rest API may not be as efficient or scalable as Bulk API