2018-06-16 16:03:37

icp(index condition pushdown)

icp(acronym of index condition pushdown), which used when index(part of a multicolumn index) can be use to filter out some(enormous maybe) on storage level

  • the output of EXPLAIN cound display some information to indicate user, suck as key_len and key, especially in Extra column which will dislay using index condition
  • I think that only multicolumn index could make use of this method
  • even there exist only one where condition, the icp method would happened, becuase it did
  • need satisfy leftmost prefix
  • For InnoDB tables, ICP is used only for secondary indexes, The goal of ICP is to reduce the number of full-row reads and thereby reduce I/O operations. For InnoDB clustered indexes, the complete record is already read into the InnoDB buffer. Using ICP in this case does not reduce I/O.
  • Conditions that refer to subqueries cannot be pushed down.
  • it depends on optimizer and analyse according to statistic to decide where use icp on specific query
Read More »
2018-06-16 16:02:17

index merge

theory

index merge is that mysql sometimes(depends on some) could use seperate index on where condition at storage level, then can scan them simultaneously and them merge the result

  • where clause could affect optimizer whether to use index merge method, especially when intersect on index merge method
  • whether to use the method, I think the cardinatity of a index is very important, the higher the cardinatity the lower the posibility to use the method, because the purpose index merge is to reduce reduncdant rows retrieve, if optimizer think it can not satify this requirement, why use it?
  • the access method merges index scans from a single table; it does not merge scans across multiple tables.
  • type, key, key_len, rows will dispaly different in EXPLAIN when optimizer decide to use this method
  • it's more easier to judge a or opration whether the optimizer decide to use this method than a and opration
Read More »
2018-06-16 16:00:05

a little about query of mysql

There some theory about query on mysql we should know so that we can use it more efficiently.

some important theories

  • avoid reduncdant data access such as appliction retrieving more data than nedd and server level get reduncdant data
  • sometimes a fewer queries would be better than a single query
  • a scan was used based on whether the best index spanned more than 30% of the table, but a fixed percentage no longer determines the choice between using an index or a scan. This indicate whether use the index to get data you want is detemined by optimizer and the data on the table
Read More »
2018-06-16 15:52:19

index in mysql

There are some types of index which depends on storage engine. Most used index would be B-Tree indexes, which complement by InnoDB.

B-Tree indexes

  1. InnoDB refers to them by their primary key values
  2. store in order
  3. multicolumn index should satisfy leftmost prefix
  4. the index can be used by ORDER BY,GROUP BY and DISTINCT
Read More »
2018-01-17 17:19:17

vim打造python ide,只看这一篇就够

前言

由于日常用macbook开发,也不用鼠标,工作内容是全python环境,用pycharm一天下来手酸的很,刚好同事提到了说之前有个家伙用vim看linux内核,才猛然想起自己大学时候也玩过这玩意,后面就不了了之了。

想了一下发现vim还挺适合我现在的工作模式,简直一拍即合,花了几天业余时间看了些教程和必要的快捷键,磕磕碰碰的把环境搭起来了,然后在公司以龟速的工作效率适应了两三天之后pycharm已经被我放角落里了。一天下来除了看资料也用不到触控板,怎一个爽字了得。

网上的绝大部分资料都不能开箱即用,不是太老就是不适合python,要不就是只有零零散散的,这篇文章的宗旨就是看完这篇配置就能完全搭起来一个自己日常使用的的python开发ide,而不是一个装逼货。当然个人用户习惯不一样,你可以根据自己的情况来配置。

本篇只是教你怎么配置(理论上debian系可以直接跟着命令敲),而不是教你怎么使用,如果有需要使用教程的可以email给我,我尽快写一个使用教程(如果你是一个使用vim的人,估计也不需要使用教程,前段时间垃圾留言太多了,也没空理它,索性把评论功能去掉了~)

先放个图吧~

Read More »
2017-12-18 19:59:26

tornado源码浅析2-TCP网络IO处理

看一个框架我喜欢先从网络方面分析,然后再来分析它是怎么达到并发/并行的效果的,所以这里先看看tornado的网络处理

为了清晰,这里就先从tornado官方的demo入手分析吧。在demos/tcpecho文件夹下面有三个文件,我们看其中的client.pyserver.py

Read More »
2017-12-17 20:36:03

tornado源码浅析1-大纲

tornado的源码其实算是比较少的了,可能是因为不像c系静态语言那样很多轮子要自己造吧。

由于公司使用的是tornado,而且业务量也不小,依据以前用skynet的经验来看,这种框架不知道实现机制的话是很危险的。 所以看看它的源码自然提上日程了,原本以为会有很多黑魔法,没想到啥黑魔法都没有,勉强算得上黑魔法的东西可能就是那个Configurable类了

本以为要花很长时间才能看懂,没想到只花了一天多一点就把机制弄的差不多了。这里先从总体上说说。

tornado的框架我把它分为两大块:

  1. IOStream,它提供数据支撑,比如网络数据的读取都在这里了
  2. IOLoop, 它配合python的生成器提供了真正的协程语义,并且它还处理epoll/kqueue/select

所以看它的源码也是从这两大块来看,这里先说先基础的东西。

Read More »
2017-12-10 09:45:28

python的property(描述器descriptor相关知识)

今天cookbook,发现里面的property看的有点懵,所以去了解了下它的实现,看了以后就感觉挺简单的,只是灵活性太高,有点绕而已

其实它使用了描述器来实现功能,所以知道什么是描述器就好办了。

Read More »