Tuesday, April 6, 2021

GTE stock research: Colombia Is Preparing For A Fracking Boom

 April 6, 2021

Here is the article. 

I like to highlight something new to me. I invested over 10,000 shares of GTE.TO. 


Editor OilPrice.com
·6 min read

It was roughly a decade ago when hydraulic fracturing, known as fracking, was identified as a crucial means for boosting Colombia’s meager hydrocarbon reserves and production. A shortage of proved crude oil and natural gas reserves along with declining production is threatening Colombia’s economy. Despite only possessing two billion barrels of proved oil reserves with a short six-year production life, Colombia is highly reliant upon petroleum production to drive the economy. During 2020, at the height of the pandemic and oil price crash, petroleum accounted for 28% of exports by income, 3% of gross domestic product, and 17% of fiscal revenue. A dearth of significant conventional oil discoveries since the late 1990s means Colombia must find alternate means of boosting crude oil and natural gas reserves if its hydrocarbon-dependent economy is to grow. In fact, a lack of significant discoveries combined with weak natural gas production due to rising decline rates at aging offshore fields and growing demand forced Bogota to start importing liquified natural gas in 2017. This shortage triggered a crisis that challenged the Andean country’s energy security, forcing the national government to work feverishly to resolve the crisis by implementing incentives aimed at promoting exploration and development activity, including paying higher than market natural gas prices at the wellhead.

In 2018, when President Ivan Duque was championing fracking as means of addressing Colombia’s lack of hydrocarbon reserves, the Council of State, the country’s highest administrative tribunal, placed a moratorium on the controversial technique. That essentially prevented the introduction of fracking in Colombia as a means of extracting crude oil and natural gas. In 2019 the tribunal upheld the ban on fracking but roughly a week after announcing that decision clarified it did not include pilot projects. A 2020 bid to block fracking pilots failed and in September that year the National Hydrocarbon Agency (ANH – Spanish initials) finalized the regulations for selecting pilot participants. Colombia’s fracking pilots are restricted to be conducted in the Middle Magdalena Valley and Cesar-Ranchería Basins. The head of the ANH, Armando Zamora, in a March 2021 Reuters article, stated he expects the 2018 moratorium on fracking in Colombia to be lifted during 2022 when environmental regulations are modified.

The Middle Magdalena Valley Basin, located in central Colombia between the eastern and central Andes mountain ranges, has over 40 discovered conventional oilfields with the U.S. Geological Service (USGS) predicting it holds undiscovered oil resources of up to 1.4 billion barrels.

EIA
EIA

Source: U.S. EIA.

Since 1918, when crude oil was first discovered in Colombia at the La Cira-Infantas oilfield in the basin, it has produced most of Colombia’s oil. The key fracking target in the Middle Magdalena Valley Basin is the La Luna geological formation which has been compared to the prolific U.S. Eagle Ford shale play. It is a Cretaceous age geological formation typified as a classic shale-gas system in which the rock is the source, reservoir, and seal. The USGS estimated in 2017 that the La Luna possesses undiscovered shale oil resources of up to 521 million barrels and over two trillion cubic feet of natural gas. It has been predicted that the Middle Magdalena Valley basin, in total, contains unconventional crude oil resources of up to seven billion barrels, or more than triple Colombia’s current proved reserves and 13 trillion cubic feet of natural gas. Those copious hydrocarbon resources, if proven to be commercially viable to extract, will resolve the significant economic downside posed by Colombia’s limited crude oil and natural gas reserves.

The push to bring fracking pilots online is gaining considerable momentum after a fraught 2020 where significant risks, including sharply weaker oil prices and continued community opposition, indicated they may never eventuate. National oil company Ecopetrol, which is 88.49% owned by the Colombian government, launched the Kale fracking pilot in the Middle Magdalena Valley in the department of Santander near the town of Puerto Wilches. The national oil company has budgeted almost $77 million to develop the project, with drilling scheduled to commence during the second half of 2021. Earlier this month global oil supermajor ExxonMobil also announced it had submitted a proposal for a fracking project in the Middle Magdalena Valley near Puerto Wilches. Exxon has earmarked $57 million for the project, which has yet to be approved by the ANH.

Related: Houthi Rebels Launch Missile Attack On Saudi Oil Terminal

The economic importance of developing fracking in Colombia cannot be emphasized enough. The Andean country’s peak petroleum industry body, the Colombian Petroleum Association (ACP – Spanish initials), believes commercial fracking in the Andean country could eventually add 450,000 barrels of daily production to current production and attract $5 billion of investment. That will provide an important source of revenue for a cash-strapped national government still reeling from the pandemic’s severe economic fallout and crude oil’s prolonged price slump. Bogota is struggling to restart a stalled economy which, despite being projected to grow by 4% during 2021 compared to contracting by 7.8% in 2020, will see the budget deficit blowout by almost one percent year over year to 8.6% of GDP. Unemployment is still alarmingly high with DANE, the national statistical agency, reporting it reached 17.3% in January 2021 the highest since July 2020 at the peak of Colombia’s pandemic lockdown. Those worrying numbers indicate that Colombia is still unable to shake off the profound economic aftermath of the pandemic, indicating that Bogota needs to find ways to restart the economy.

The Andean country’s considerable economic dependence on hydrocarbon extraction makes it especially vulnerable to its limited oil and natural gas reserves, including their short production life and oil price slumps. It appears that Colombia is unable to end its addiction to extracting fossil fuels to power the economy. The Andean country’s energy minister recently stated Colombia would double down on thermal coal mining, despite it being the first fossil fuel to be phased out as part of the fight against global warming. By bolstering hydrocarbon reserves and production Bogota will only further expose the economy to volatile oil prices and the immense downside which exists once peak oil demand arises, which according to some industry analysts could be in as little as seven years. Considerable community opposition to fracking remains tremendous. That, combined with the petroleum industry’s slowly waning social license and ongoing security issues, is creating considerable uncertainty which is weighing on investment in unconventional oil and gas production in Colombia.

Monday, April 5, 2021

7.1 Incrementally Refactoring a Monolith Into Microservices Event Driven Microservices

 April 5, 2021

Here is the link. 

I will take some notes and then figure out what to learn next. 


Interview: Chris Richardson Of Microservices.io

 April 5, 2021

Here is the link. 

Chris Richardson, creator of Microsrevices.io, talks about distributed data in microservices, common pitfalls for developers and architects to avoid, and the process of moving away from the monolith.

6.7 Event Sourcing and Microservices Event Driven Microservices

April 5, 2021

Here is the link. 

Aggregates + Event sourcing = Modular domain model 

Modular domain model

tightly coupled ACID 

Event sourcing 

Microservice architecture 


6.6 Implementing Queries in an Event Source Application Event Driven Microservices

 April 5, 2021

Here is the link. 

Persisting a customer and order history in MongoDB

Denormalized = efficient lookup 

Other kinds of views

AWS cloud search

  • Text search as a service
  • View updater batches aggregate to index
  • View query service does text search
AWS DynamoDB
  • NoSQL as a Service
  • On demand scalable specify desired read/write capacity
  • Document and key value data models
  • Useful for denormalized UI oriented views
Benefits and drawbacks of CQRS
Benefits
  • Necessary in an event sourced architecture
  • Separation of concerns = simpler command and query models
  • Supports multiple denormalized views
  • Improved scalability and performance
Drawbacks
  • Complexity
  • Potential code duplication
  • Replication lag/eventually consistent views





Domain-driven design | Book reading: Design domain-driven design Eric Evens

 April 5, 2021

Domain-driven design (DDD) is the concept that the structure and language of software code (class names, class methods, class variables) should match the business domain. For example, if a software processes loan applications, it might have classes such as LoanApplication and Customer, and methods such as AcceptOffer and Withdraw.

DDD connects the implementation to an evolving model.[1]

Domain-driven design is predicated on the following goals:

  • placing the project's primary focus on the core domain and domain logic;
  • basing complex designs on a model of the domain;
  • initiating a creative collaboration between technical and domain experts to iteratively refine a conceptual model that addresses particular domain problems.

The term was coined by Eric Evans in his book of the same title.


Relationship to other ideas[edit]

Object-oriented analysis and design
Although, in theory, the general idea of DDD need not be restricted to object-oriented approaches, in practice, DDD seeks to exploit the advantages that object-oriented techniques make possible. These include entities/aggregate roots as receivers of commands/method invocations and the encapsulation of state within foremost aggregate roots and on a higher architectural level, bounded contexts.
Model-driven engineering (MDE) and Model-driven architecture (MDA)
While DDD is compatible with MDA/MDE (where MDE can be regarded as a superset of MDA) the intent of the two concepts is somewhat different. MDA is concerned more with the means of translating a model into code for different technology platforms than with the practice of defining better domain models. The techniques provided by MDE (to model domains, to create DSLs to facilitate the communication between domain experts and developers,...) facilitate the application of DDD in practice and help DDD practitioners to get more out of their models. Thanks to the model transformation and code generation techniques of MDE, the domain model can be used not only to represent the domain but also to generate the actual software system that will be used to manage it. This picture shows a possible representation of DDD and MDE combined.
Plain Old Java Objects (POJOs) and Plain Old CLR Objects (POCOs)
POJOs and POCOs are technical implementation concepts, specific to Java and the .NET Framework respectively. However, the emergence of the terms POJO and POCO reflect a growing view that, within the context of either of those technical platforms, domain objects should be defined purely to implement the business behaviour of the corresponding domain concept, rather than be defined by the requirements of a more specific technology framework.
The naked objects pattern
Based on the premise that if you have a good enough domain model, the user interface can simply be a reflection of this domain model; and that if you require the user interface to be a direct reflection of the domain model, then this will force the design of a better domain model.[4]
Domain-specific modeling (DSM)
DSM is DDD applied through the use of Domain-specific languages.
Domain-specific language (DSL)
DDD does not specifically require the use of a DSL, though it could be used to help define a DSL and support methods like domain-specific multimodeling.
Aspect-oriented programming (AOP)
AOP makes it easy to factor out technical concerns (such as security, transaction management, logging) from a domain model, and as such makes it easier to design and implement domain models that focus purely on the business logic.
Command Query Responsibility Segregation (CQRS)
CQRS is an architectural pattern for separation of reads from writes, where the former is a Query and the latter is a Command. Commands mutate state and are hence approximately equivalent to method invocation on aggregate roots/entities. Queries read state but do not mutate it. CQRS is a derivative architectural pattern from the design pattern called Command and Query Separation (CQS) which was coined by Bertrand Meyer. While CQRS does not require DDD, domain-driven design makes the distinction between commands and queries explicit, around the concept of an aggregate root. The idea is that a given aggregate root has a method that corresponds to a command and a command handler invokes the method on the aggregate root. The aggregate root is responsible for performing the logic of the operation and yielding either a number of events or a failure (exception or execution result enumeration/number) response OR (if Event Sourcing (ES) is not used) just mutating its state for a persister implementation such as an ORM to write to a data store, while the command handler is responsible for pulling in infrastructure concerns related to the saving of the aggregate root's state or events and creating the needed contexts (e.g., transactions).
Event Sourcing (ES)
An architectural pattern which warrants that your entities (as per Eric Evans' definition) do not track their internal state by means of direct serialization or O/R mapping, but by means of reading and committing events to an event store. Where ES is combined with CQRS and DDD, aggregate roots are responsible for thoroughly validating and applying commands (often by means of having their instance methods invoked from a Command Handler), and then publishing a single or a set of events which is also the foundation upon which the aggregate roots base their logic for dealing with method invocations. Hence, the input is a command and the output is one or many events which are transactionally (single commit) saved to an event store, and then often published on a message broker for the benefit of those interested (often the views are interested; they are then queried using Query-messages). When modeling your aggregate roots to output events, you can isolate the internal state even further than would be possible when projecting read-data from your entities, as is done in standard n-tier data-passing architectures. One significant benefit from this is that tooling such as axiomatic theorem provers (e.g., Microsoft Contracts and CHESS[5]) are easier to apply, as the aggregate root comprehensively hides its internal state. Events are often persisted based on the version of the aggregate root instance, which yields a domain model that synchronizes in distributed systems around the concept of optimistic concurrency.

Vancouver IT companies

 如果你决定留在温哥华, 那么求稳去知名上市公司的. 本地office有些工作经验的可以尝试, remote然而个人认为兼顾成长收入和稳定的选择应该是加入美国中后期startup公司在温哥华的office或者remote

上市公司就不多说了, Amazon主要是招的人多, Microsoft第二, Apple, Facebook, Google, Oracle招的人是真少, 其他的不错的有 Salesforce, Tableau, Square, Slack, Splunk, Segment(Twilio), Asana, Shopify, Workday, Mastercard, 基本错不了, Cisco和SAP福利不错Fortinet, GEdigital, Broadcom, Move(Realtorcom), Arista据说还可以

如果可以接受remote, 那选择是相当之多. 包括Twitter, coursera, Wish, Intuit, SurveyMonkey, MongoDB等. 

Startup

在温哥华开office有很多无法一一列举处于中后期的有Dialpad, Chime, Postmates

其他startup中比较有名的有Flipboard, Mozilla, Kickstarter, Zenefits已被收购单论工资还不错

除此之外温哥华本地也有很多Startup, 像Slack, 原本就是温哥华发家的. 

本地startup中做的比较好的有Hootsuite, Visier, Clio, Trulioo, Axiom, Zen(CryptoKitty/NFT), Tasktop, D-Wave(QuantumComputer), 饭团, POF等.

如果可以接受remote那么其实优秀的startup相当之多Esty, Stripe, Instacart, Brex, Okta, Coinbase, Zapier, Gusto, PagerDuty等

温哥华游戏公司也挺多, 不过总的来说性价比不高. 钱不多事不少, 大一些的有EA, Demonware(ActivisionBlizzard), Relic, Skybox, IUGO, Kabam, TheCoalition(GearsofWar)

小的就更多了, Archiact, A ThinkingApe, Klei, EastSideGames

数不过来其实Capcom, Nintendo, Bandai, Namco, KOEI.

在温哥华都有办公室但是招人非常少-baidu, 1point3acres

温哥华还有一些公司不完全符合上述条件但是机会也不错比如FinTech方面有RBC的lab, Borealis AI高频交易公司, DRW应该都是不差钱的. 

至于Shaw, Telus, Bell之类的就不太熟悉了.

总的来说温哥华的收入和美国相比要低不少. 

这是事实, 然而因为温哥华和硅谷处于同一时区, 所以这样的势能差逐渐被很多公司发现并利用. 

加上政府的优惠税收政策和宽松的移民政策使得这几年到温哥华来开office的公司越来越多. 

此外疫情也是一个相当重要的因素, 很多公司意识到了remote是完全可行的. 所以remote的机会也越来越多.

以上这些因素导致了温哥华码农这几年的待遇逐步提高并且长期趋势看好. 当然无论如何南边的机会肯定更多是没错的

Pessimistic lock | Optimistic lock | Learning in Chinese

I will review the content to make it readable.  

悲观锁( pessimistic lock  ): 

通常是由数据库机制实现的,在整个过程中把数据锁住(查询时),只要事务不释放(提交或回滚),任何用户都不能查看和修改。锁主要是解决并发性问题。 

通过 jdbc  实现时 sql  语句只要在整个语句之后加 for update  即可。例如: select …for update  

Hibernate  中加载一个持久化类时需要用 load  方法  这个方法是对普通的 load  的重载。 

如: Student inv=  ( Student     ) session.load(Student   .class,”1001”,LOCKMODE.UPGRADE);  

这时需要说明一个问题,这样就加了锁锁的类型可以根据 LOCKMODE  的值决定。加了锁之后, Hibernate  的 lazy  策略就失效了,此时就发了 sql  语句,因为需要把这个对象加载进来锁住,所以发了 SQl  语句。当有一个方法通过悲观锁机制加载某个对象的时候,对这个对象进行了一系列的操作,在进行操作的时候,也就是只要事务未提交,这个锁就一直存在。当另外一个方法加载这个对象的时候(两个对象是一个对象,即唯一标示符的值是相同的)只会发出查询语句,停止不动,因为,前一个方法使用了悲观锁机制加载的这个对象,并没有结束事务(提交或回滚),因此这时是排他的。当第一个方法提交了事务,第二个方法才可以加载成功并按照自己的意愿执行其所有操作。 

悲观锁的使用:悲观锁解决了更新丢失( lost update  )问题,但是也带来了并发问题 à  并发不好。但是,如果在某个需求中,需要很多,很频繁的并发操作,特别是,某个操作或事务又会占用很多时间时,其他操作只能静而后之了。要看具体应用了。 

Hibernate  的加锁模式有:   

  LockMode.NONE   :   无锁机制。   

  LockMode.WRITE   : Hibernate  在 Insert  和 Update  记录的时候会自动 

获取。 
  LockMode.READ   : Hibernate  在读取记录的时候会自动获取。   

以上这三种锁机制一般由 Hibernate  内部使用,如 Hibernate  为了保证 Update   

过程中对象不会被外界修改,会在 save  方法实现中自动为目标对象加上 WRITE  锁。   

  LockMode.UPGRADE   :利用数据库的 for update  子句加锁。   

  LockMode. UPGRADE_NOWAIT   : Oracle  的特定实现,利用 Oracle  的 for   

update nowait  子句实现加锁。 

乐观锁( optimistic  ): 

乐观锁其实不是一种锁,也就不是锁住的问题,而是给数据库表加入了一个字段(可以使版本号( version  ),也可以使一个时间戳( timestamp  )),或是进行全部字段 /  脏数据字段比较(这种方式适合于以前遗留下来的系统,在不更改原来表结构的时候使用这种策略)来确定数据是否被修改过,一般的应用是采用数据版本的方式( version  )实现,在读取数据的时候将 version  读取出来,在保存数据的时候判断 version  的值是否小于数据库中的 version  的值,小于则不允许更新,否则可以更新。 

使用 version  实现乐观锁(推荐使用):在一个事务提交后就会更改数据库,数据库中 version  的值会自动加 1  。实现步骤   1  、在持久化类中加入 version  属性,生成其 getter  和 setter  方法。 2  、在配置文件中的 <class>  标签中配置一个属性 optimistic-lock=”version”  (这个属性的默认值就是 version  ,可以不进行配置,但建议配置上) 3  、对 version  字段进行映射,使用 version  标签(这个字段的映射必须在 id  标签的后面第一位) <version   name=”version”/>  

细节分析 在一个事务加载某个持久化类时,对这个对象进行了一系列操作,但是还没有提交事务,于此同时,另外一个事物也加载了这个持久化类,并完成了一系列的操作后提交了事务,然后,第一个事务这时也要提交事务了,这样就会抛出一个异常   org.hibernate.StableObjectStateException:Rows was updated or deleted by another transaction…  ,原因是这样的:当一个事务提交时会发出这样一条 SQL  语句 update   表名 set   所有表属性 =? where id(  唯一标示符值 )=? and version=?    这个语句中 version=?   是最关键的。以上说的那个例子中,第一个事务拿的是自己的旧的 version  值,进行更更新,而第二个事务在提交后,已经改变了 version  ,变成了新的 version  值了,这样第一个事务提交事务时发出的 update   表名 set   所有表属性 =? where id(  唯一标示符值 ) =? and version=?    这条语句将会失败!就会抛出以上异常信息。 

乐观锁:适合于高并发。 

optimistic-lock  属性有如下可选取值:  

    none   

无乐观锁  

  version  

通过版本机制实现乐观锁  

  dirty  

通过检查发生变动过的属性实现乐观锁  

  all  

通过检查所有属性实现乐观锁

 


ACAD stock: Hard working can make profit $900 difference in a day



 

CTRM stock: over 20% loss | Best analytical analysis | Best article for beginner

 CTRM stock 



System design: Microservice.io - Chris Richardson | Transaction Log Tailing With Debezium — Part 1

 April 5, 2021

Here is the article on medium.com. 

Sometimes there can be more than one action we need to take in any business logic, especially when coding a business on the backend development side. There may be event-driven actions such as publishing a message to any message broker in these actions to be taken.

As an example, let’s assume that we receive the information that the customer returns are approved and processed in the database in a return process. We also need to leave a message in a message broker system to pay the money back. 


Failure to send the event is possible. What should we do to avoid such a situation? The Transactional Outbox pattern comes into play right at this point.

This model provides an effective solution for reliably publishing events. The purpose of this approach is to have an “Outbox” table in the database of the service. When receiving the registration request, not only addition is made to the ReturnOrder table, but also a record representing the event is added to the Outbox table.

An asynchronous process monitors the Outbox table for new entries and publishes any event on the Event Bus. The model improves reliability by simply dividing two processes on different services.

Polling Publisher

The Polling Publisher approach is a structure built to publish the records in the database’s Outbox table in order. A scheduled job that runs periodically processes the records in the Outbox table, then deletes the corresponding record. This structure is easy and costless to implement in low-scale systems. However, it also has some disadvantages.

  • Publishing events in order is difficult.
  • As the number of calls to the database increases as the scale increases, it will cause a performance problem after a while.
  • Not applicable in all NoSQL databases.
Using the outbox pattern with polling publisher approach

Transaction Log Tailing

This is another option to access distributed transactions. The idea is to queue the database transaction log and publish each change as an event. Each committed update is displayed as an entry in the transaction log of the database. A transaction log miner can read the transaction log entry and publish each change as a message to Message Broker. Converts each relevant log entry corresponding to the added data into a message. The messages are then published in a Message Broker. The benefit of this model is that there is no need for any changes at the application level; everything happens at the database level. It is a little difficult to avoid duplicate publishing. Therefore, in order to ensure data consistency, the application that will consume the related events must be idempotent.

  • Transactions take place in order.
  • The accuracy of the data is guaranteed.
  • No extra calls are made to the database through an application.
  • It can be applied in high scale systems.
Using the outbox pattern with transaction log tailing approach

More detailed information about the Transactional Outbox pattern is available on Chris Richardson’s microservices.io site.

Sunday, April 4, 2021

6.2 Using Events to Maintain Data Consistency Event Driven Microservices

 April 4, 2021

Event-driven order processing 

Eventually consistent credit checking 

Place order, order created, credit reserved, or credit check failed 

Order management   customer management 

Problem #1: How to design eventually consistently? 

Problem #2: How atomically update database and publish an event 

Failure = inconsistent system 

Order service 

Update and publish using 2PC 

  • Guaranteed atomicity BUT
  • Need a distributed transaction manager
  • Database and message broker must support 2PC
  • Impacts reliability
  • Not fashionable
  • 2PC is best avoided 

4.2 Communication Patterns API Gateway Event Driven Microservices

April 4, 2021

Here is the link. 

Forces

  • Mismatch between the fine-grained microservices and needs of the clients
  • Different clients need different data
  • LAN vs. WAN vs. Mobile network performance
  • The number of service instances and their locations (host + port) changes dynamically
  • Partitioning into services can change over time and should be hidden from clients. 
API gateway pattern 

single entry point - > API Gateway 

Client specific APIs, protocol translation 
AMQP - websocket message - > 
Device specific end points